Caching mechanism of django

django setup cache

Setting up Cache

  • Django supports database-based, file-based and memory-based caching. Usually we have to set it up first. Django's caching settings are located in the CACHES configuration item in settings.py.

  • Django supports the following caching systems:

1. Memcached

  • Memcached is a caching system originally supported by Django, which is fast and efficient. Memcached is a memory-based caching service, originally developed to address the load problem of LiveJournal.com social networking sites, and later opened by Danga. It is used by large websites like Facebook and Wikipedia to reduce the number of database visits and significantly improve the performance of the site.
  • Memcached starts a daemon and allocates separate memory blocks. Its main work is to provide a fast interface for adding, retrieving and deleting caches. All data is stored directly in memory, so it cannot replace the functions of database or file system. If you're familiar with caching, it's easy to understand.

If you are a novice, be clear:

  • Memcached is not Django's own software, but an independent software that requires you to install, configure, and start services yourself.
  • When Memcached is installed, Python's dependency libraries for operating Memcached are also installed. The most commonly used are python-memcached and pylibmc.
  • Once both of the above conditions are met, they will be configured in Django.

Configuration method:

  • Depending on the Python dependency libraries you install, set CACHES BACKEND to

    django.core.cache.backends.memcached.MemcachedCache
    //Or django.core.cache.backends.memcached.PyLibMCCache
  • Set up LOCATION for the host IP and process port where your Memcached daemon is located, in the form of a string of ip:port. Or in the form of unix:path, in the Unix operating system.

  • Here's a reference example. Memcached runs on localhost (127.0.0.1) port 11211, using the python-memcached library:

    CACHES = {
        'default': {
            'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
            'LOCATION': '127.0.0.1:11211',
        }
    }
  • The following Memcached runs on the local Unix socket: / tmp/memcached.sock, depending on python-memcached:

    CACHES = {
        'default': {
            'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
            'LOCATION': 'unix:/tmp/memcached.sock',
        }
    }
  • The following Memcached runs at / tmp/memcached.sock, without unix:/ prefix, and relies on the pylibmc library:

    CACHES = {
        'default': {
            'BACKEND': 'django.core.cache.backends.memcached.PyLibMCCache',
            'LOCATION': '/tmp/memcached.sock',
        }
    }
  • Memcached supports distributed services and may run on several machines at the same time, adding their IP addresses to LOCATION, as follows:

    CACHES = {
        'default': {
            'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
            'LOCATION': [
                '172.19.26.240:11211',
                '172.19.26.242:21423',
                '172.19.26.244:11213',
            ]
        }
    }

The obvious disadvantage of memory-based caching system is the loss of power-off data, especially Memcached, which does not support serialization, so you must pay attention to the security of data.

In fact, for the moment, redis is better as a cache and also supports serialization.

2. Database cache

  • The main reason why we use caching is to reduce the operation of the database. If we store the caching in the database again, we do no work. So, try not to use database-based caching, nor do I introduce it here. Give me a simple configuration example.

    CACHES = {
        'default': {
            'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
            'LOCATION': 'my_cache_table',  # Database table name
        }
    }
    # Note: Execute the command Python management.py createcachetable for creating cached tables 

But in some cases, there are still some uses, such as you have a high-speed, efficient indexed database.

redis cache

  • Preparatory software: django-redis third-party module

  • Start using:

  • Installation: pip install django-redis

  • To configure: settings.py

    CACHES = {
        "default": {
            "BACKEND": "django_redis.cache.RedisCache",
            "LOCATION": "redis://127.0.0.1:6379/1",
            "OPTIONS": {
               "CLIENT_CLASS": "django_redis.client.DefaultClient",
            }
        }
    }
  • Use configuration as session backend

    SESSION_ENGINE = "django.contrib.sessions.backends.cache"
    SESSION_CACHE_ALIAS = "default"

3. File system caching

  • Even databases are slow, so what about file systems? More slowly! But when you don't have Redis, Memcached, and databases in your hands, you can use them reluctantly. Here are two configuration cases:

  • Unix-based system:

    CACHES = {
        'default': {
            'BACKEND': 'django.core.cache.backends.filebased.FileBasedCache',
            'LOCATION': '/var/tmp/django_cache',
        }
    }
    
  • Based on Windows operating system, the path with disc symbol is needed:

    CACHES = {
        'default': {
            'BACKEND': 'django.core.cache.backends.filebased.FileBasedCache',
            'LOCATION': 'c:/foo/bar',
        }
    }
    

4. Local memory-based caching

  • If your local host memory is large enough and fast enough, you can also use it directly as a cache. The configuration is as follows:

    CACHES = {
        'default': {
          'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
          'LOCATION': 'unique-snowflake',
             # Cache timeout time (default 300, None means never expired, 0 means immediate expiration)
          'TIMEOUT': 300, 
          'OPTIONS':{
                      # Maximum number of caches (default 300)
                        'MAX_ENTRIES': 300,   
                      # When the maximum number of caches is reached, the ratio of the number of caches is eliminated, i.e. 1/CULL_FREQUENCY (default 3)
                        'CULL_FREQUENCY': 3,  
                    },
              } 
          }
    
    # Cache view
    from django.views.decorators.cache import cache_page@cache_page(5)  # Parameter timeout time, expired after 5 seconds
    def student_list(request, *args, **kwargs):  
        students = models.Student.objects.all()  
        print('students')  
        return render(request, 'student_list.html', {'students': students})
    

5. Developing cache for debugging

  • Django is very considerate in designing a development cache for us. When your production environment is a large caching system, and you are developing without the corresponding caching system support, or don't want to use that kind of bulky big guys to develop. But in the actual development process, you have to access the caching system and use the caching api. In this case, the development of caching is easy.

  • The configuration is as follows:

    CACHES = {
        'default': {
            'BACKEND': 'django.core.cache.backends.dummy.DummyCache',    
            'TIMEOUT': 300, # Cache timeout time (default 300, None means never expired, 0 means immediate expiration)
         'OPTIONS':{
                      # Maximum number of caches (default 300)
                        'MAX_ENTRIES': 300,   
                      # When the maximum number of caches is reached, the ratio of the number of caches is eliminated, i.e. 1/CULL_FREQUENCY (default 3)
                        'CULL_FREQUENCY': 3,  
                    },
          'KEY_PREFIX': '',        # Cache key prefix (default empty)
             'VERSION': 1,            # Cache version of key (default 1)
             'KEY_FUNCTION': Function name   # Functions that generate keys (default functions are generated as: [prefix: version: key]
        }
    }
    

6. Custom Cache Backend

  • The highest level of course is to use the caching system developed by yourself, Django is supported, but the premise is that you have that ability! The configuration is simple:

    CACHES = {
        'default': {
            'BACKEND': 'path.to.backend',
        }
    }
    

7. Caching parameters

  • Each of the aforementioned caching backends can set some additional parameters to control the caching behavior. The parameters can be set as follows:
    • TIMEOUT: The default expiration time of the cache is in seconds, and the default is 300 seconds None to indicate that it will never expire. Setting to 0 causes the cache to fail immediately (the cache is meaningless).
    • OPTIONS: Optional parameters, depending on the backend of the cache.
    • KEY_PREFIX: Strings of all cached keys used by the Django server.
    • VERSION: The default version number generated by the Django server.
    • KEY_FUNCTION: A string containing the point path of a function that defines how to combine prefixes, versions, and keys into the final cache key.
  • The following example configures a file system-based cache backend with a cache expiration time set to 60 seconds and a maximum entry of 1000.

    CACHES = {
        'default': {
            'BACKEND': 'django.core.cache.backends.filebased.FileBasedCache',
            'LOCATION': '/var/tmp/django_cache',
            'TIMEOUT': 60,
            'OPTIONS': {
                'MAX_ENTRIES': 1000
            }
        }
    }
    
  • The following example configures a backend based on python-memcached library with an object size limit of 2MB:

    CACHES = {
        'default': {
            'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
            'LOCATION': '127.0.0.1:11211',
            'OPTIONS': {
                'server_max_value_length': 1024 * 1024 * 2,
            }
        }
    }
    
  • The following is a back-end configuration based on the pylibmc library, which enables binary protocols, SASL authentication and ketama behavior patterns:

    CACHES = {
        'default': {
            'BACKEND': 'django.core.cache.backends.memcached.PyLibMCCache',
            'LOCATION': '127.0.0.1:11211',
            'OPTIONS': {
                'binary': True,
                'username': 'user',
                'password': 'pass',
                'behaviors': {
                    'ketama': True,
                }
            }
        }
    }
    

2. Total Station Application Cache

  • The easiest way to use the caching system is to cache the entire website.

  • This requires the addition of'django.middleware.cache.UpdateCacheMiddleware'and'django.middleware.cache.FetchFromCacheMiddleware' to the MIDDLEWARE settings as follows:

    MIDDLEWARE = [
        'django.middleware.cache.UpdateCacheMiddleware',  # Must be at the top  
        # Other middleware...
        'django.middleware.cache.FetchFromCacheMiddleware', # Must be at the end]
    
  • Note: Update middleware must be at the beginning of the list, while fectch middleware must be at the end. This is Django's rule for using middleware. They are sequential and executed from top to bottom. When the request comes, execute the middleware process UU request method first. At this time, you need to first see whether there is data in the cache. In the effect middleware, there is a process UU request method, which is used to obtain the cache data. If there is any, the route matching will not be performed, and the data will be returned directly, so you need to put it at the end. When the data is going to be returned to the browser, update the cache and put the data in the cache, so there is a process_response method in the update middleware to update the cached data, so put it at the top.

  • Then, add the following required parameters to the settings file:

    CACHE_MIDDLEWARE_ALIAS: Alias for Cache for Storage
     CACHE_MIDDLEWARE_SECONDS: How many seconds does each page need to be cached?
    CACHE_MIDDLEWARE_KEY_PREFIX: Key prefix
    

3. View Application Cache

  • Another way to use the caching framework is to cache the output of the view. In django.views.decorators.cache, a decorator cache_page that automatically caches the response results of views is defined. It is very simple to use:

    from django.views.decorators.cache import cache_page
    
    @cache_page(60 * 15)
    def my_view(request):
        ...
    

    cache_page takes a parameter: timeout, in seconds. In the previous example, the results of my_view() view will be cached for 15 minutes (written to 60 * 15 to improve readability)

  • Like site caching, view caching has nothing to do with URLs. If multiple URLs point to the same view, each URL will be cached separately. Continue with my_view example if the URLconf is as follows:

    urlpatterns = [
        url(r'^foo/([0-9]{1,2})/$', my_view),
    ]
    

    The requests sent to / foo / 23 / and / foo/1 / will be cached respectively. However, once a specific URL (e.g. / foo/23/) has been requested, the subsequent request to that URL will use the cached content.

  • cache_page can also use some additional parameters, such as cache, which indicate the specific cache back end to be used.

    @cache_page(60 * 15, cache="special_cache")
    def my_view(request):
        ...
    
  • Optional keyword parameter key_prefix can also be used to specify specific cache prefixes in each view, as follows:

    @cache_page(60 * 15, key_prefix="site1")
    def my_view(request):
        ...
    

4. Caching template fragments

  • We can also use the cache template tag to cache a fragment of the template. To use this label, first add {% load cache%} at the top of the template.

  • Template tag {% cache%} will cache the contents contained in the tag block within a set time. It requires at least two parameters: the cache time (in seconds) and the name given to the cache fragment. Like this:

    {% load cache %}
    {% cache 500 sidebar %}
        .. sidebar ..
    {% endcache %}
    
  • Multiple versions can also be cached based on the dynamic content within the fragment. As in the previous example, different versions of sidebar caches can be generated for each user of the site. Simply pass an additional parameter to the {% cache%} tag to identify and distinguish the cache fragment, as follows:

    {% load cache %}
    {% cache 500 sidebar request.user.username %}
        .. sidebar for logged in user ..
    {% endcache %}
    
  • The cache timeout parameter can be a template variable as long as the template variable can be interpreted as an integer value. For example, if the template variable my_timeout is set to a value of 600, the following two examples are equivalent:

    {% cache 600 sidebar %} ... {% endcache %}
    {% cache my_timeout sidebar %} ... {% endcache %}
    

Note: Source https://blog.csdn.net/weixin_39726347/article/details/88035356

Tags: Python Django Database Unix

Posted on Fri, 11 Oct 2019 20:51:40 -0700 by sufian