Varnish reverse proxy

Varnish reverse proxy

Dead work

Prepare a local machine as a test machine, three virtual machines:
server1: 172.25.24.1
server2: 172.25.24.2
server3: 172.25.24.3
Server

1. Configure reverse proxy

VCL processing
The process can be roughly divided into the following steps:
(1)Receive state, that is, the entry state of request processing. According to VCL rules, the request should be Pass or Pass.
Pipe, or go to Lookup (local query).
(2)Lookup state, after entering this state, the data will be looked up in the hash table, if found, it will enter Hit state, otherwise enter
Enter miss state.
(3)Pass state, in this state, will enter the back-end request, that is, into the fetch state.
(4)Fetch state, in the Fetch state, the request is retrieved at the back end, sent the request, obtained the data, and carried out locally.
Storage.
(5)Deliver status, send the acquired data to the client, and then complete the request.

On proxy server server 1

##Install varnish package
yum install varnish-3.0.5-l.e16.x86_64.rpm varnish-libs-3.0.5-l.e16.x86_64.rpm
##View the varnish configuration file to see its working environment
vim /etc/sysconfig/varnish 

##Declare varnish's required working environment to the system
vim /etc/security/limits.conf
varnish         -       nofile          131072
varnish         -       memlock         82000
varnish         -       nproc           unlimited

##Configuring varnish services
vim /etc/varnish/default.vcl
backend default {
  .host = "172.25.24.2";  ##Specify the default background server as 172.25.24.2
  .port = "80";				##Specify a connection port of 80
}

##Configure varnish service port
vim /etc/sysconfig/varnish
VARNISH_LISTEN_PORT=80   ##Line 66, modify port to 80

/etc/init.d/varnish restart   ##Restart varnish service


On the default background server server server 2

##Download the httpd service and set the default publishing page content to server2
yum install httpd -y
vim /var/www/html/index.html
server2:www.westos.org
/etc/init.d/httpd start

On the test machine:

##When we visited 172.25.24.1, we saw the web page of server2, indicating that the experiment was successful.
[root@foundation24 ~]# curl 172.25.24.1
server2:www.westos.org

2. View cache hits

On proxy server server 1

vim /etc/varnish/default.vcl 
sub vcl_deliver {
if (obj.hits>0){
set resp.http.X-Cache = "HIT from westos cache";
}
else {
set resp.http.X0Cache = "MISS from westos cache";
}
return (deliver);
}

/etc/init.d/varnish reload

##Cache refresh:
varnishadm ban.url .*$				    ##Clear all
varnishadm ban.url  /index.html   ##Clear index.html page cache
varnishadm ban.url /admin/$		##Clear admin directory cache

Test:
On the test machine

##You can see the cache from the X0Cache: column
##X0Cache: MISS from westos cache indicates no cache
##X-Cache: HIT from westos cache indicates that the data source is from the cache
curl -I 172.25.24.1


##Clear cache on proxy server first
curl  -I 172.25.24.1/index.html

3. Define back-end servers for different sites

Domain name reverse proxy will take you to different back-end servers

On the proxy server

vim /etc/varnish/default.vcl 
backend web1 {
  .host = "172.25.24.2";
  .port = "80";
}

backend web2 {
  .host = "172.25.24.3";
  .port = "80";
}
##When accessing the www.westos.org domain name, fetch data from web1, and when accessing the bbs.westos.org domain name, fetch data from web2.
//Error reporting when accessing other pages.
sub vcl_recv {
        if (req.http.host ~"^(www.)?westos.org") {
                set req.http.host = "www.westos.org";
                set req.backend = web1;
}elsif (req.http.host ~"^bbs.westos.org"){
                set req.backend = web2;
}else {
                error 404 "error";
}
}

/etc/init.d/varnish reload


On the background server server server 3

yum install httpd -y
vim /var/www/html/index.html
server3:bbs.westos.com

At the test end:

##Configure local parsing
[root@foundation24 ~]# vim /etc/hosts
172.25.24.1 server1 www.westos.org bbs.westos.org westos.org

##Domain name reverse proxy will take you to different back-end servers
[root@foundation24 ~]# curl westos.org
server2:www.westos.com
[root@foundation24 ~]# curl bbs.westos.org
server3:bbs.westos.com

4. Load balancing configuration:

1) What is load balancing:
Load Balance (Load Balance) means that it is allocated to multiple operating units for execution, such as Web servers, FTP servers, enterprise key application servers and other key task servers, so as to accomplish work tasks together.

2) The role of load balancing:
(1) Solve the concurrency pressure and improve the application processing performance (increase throughput, strengthen network processing capacity);

(2) Providing failover to achieve high availability;

(3) Provide website scalability (expansibility) by adding or reducing the number of servers;

(4) Safety protection; (Load balancing equipment to do some filtering, black-and-white list processing)

3) Construction of Load Balancing
On the proxy server

[root@server1 varnish]# vim /etc/varnish/default.vcl 


backend web1 {
  .host = "172.25.24.2";
  .port = "80";
}

backend web2 {
  .host = "172.25.24.3";
  .port = "80";
}

director lb round-robin {  ###Aggregate multiple backends into a group to achieve rotation
        { .backend = web1;}
        { .backend = web2;}
}


sub vcl_recv {
        if (req.http.host ~"^(www.)?westos.org") {
                set req.http.host = "www.westos.org";
                set req.backend = lb;
                return (pass);  ##Do not record the cache, open it when testing, but record the cache to ease the pressure of the background server when actually working.
}elsif (req.http.host ~"^bbs.westos.org"){
                set req.backend = web2;
}else {
                error 404 "error";
}
}

On server 3 server

##Configure two httpd virtual hosts and set two virtual hosts to publish page information by default
[root@server3 html]# mkdir /www
[root@server3 html]# cd
[root@server3 ~]# cd /www/
[root@server3 www]# vim index.html
server3:www.westos.org

[root@server3 www]# mkdir /bbs
[root@server3 www]# cd /bbs/
[root@server3 bbs]# vim index.html
server3:bbs.westos.org
##Turn on the httpd virtual host function
[root@server3 bbs]# vim /etc/httpd/conf/httpd.conf 
 990 NameVirtualHost *:80  ##Turn on the virtual host

1003 <VirtualHost *:80>
1004     DocumentRoot /www
1005     ServerName www.westos.org
1006 </VirtualHost>
1007 <VirtualHost *:80>
1008     DocumentRoot /bbs
1009     ServerName bbs.westos.org
1010 </VirtualHost>

[root@server3 bbs]# /etc/init.d/httpd restart

On the test machine:

##Visit www.westos.org to see that the servers visited rotate between server 2 and server 3

[root@foundation24 ~]# curl www.westos.org
server2:www.westos.org
[root@foundation24 ~]# curl www.westos.org
server3:www.westos.org
[root@foundation24 ~]# curl bbs.westos.org
server3:bbs.westos.org

5.varnish cdn push platform

On proxy server server 1:

##Download php, uzip, http services and extract the push platform compressed package into the http shared directory
[root@server1 ~]# yum install unzip php httpd -y
[root@server1 ~]# unzip bansys.zip -d /var/www/html/
[root@server1 ~]# cd /var/www/html/
[root@server1 html]# mv bansys/* .

##Edit the php file, leaving only the following settings, with the rest commented out
[root@server1 html]# vim config.php 

<?php
 //varnish host list
 //Multiple host lists can be defined
 $var_group1 = array(
                        'host' => array('172.25.24.1'),
                                                'port' => '8080', 
                    );

 //varnish group definition
 //Binding host list
 $VAR_CLUSTER = array(
                         'www.westos.org' => $var_group1,
                     );


 //varnish version
 //2.x and 3.x push commands are different
 $VAR_VERSION = "3";

?>
##bansys has two working modes: telnet and http.
##telnet mode needs to close the validation of varnish service management port and comment out "-S" in / etc/sysconfig/varnish file$
{VARNISH_SECRET_FILE}"This line,restart varnish Service is enough.
##If it's http mode, you need to set varnish as follows:
[root@server1 html]# vim /etc/varnish/default.vcl 

acl westos {
#Setting Access Control
"127.0.0.1";
"172.25.24.0"/24;
}


backend web1 {
  .host = "172.25.24.2";
  .port = "80";
}

backend web2 {
  .host = "172.25.24.3";
  .port = "80";
}

director lb round-robin {
        { .backend = web1;}
        { .backend = web2;}
}

sub vcl_recv {
        if (req.request == "BAN") {
                if (!client.ip ~ westos) {
                        error 405 "Not allowed.";
                }
                ban("req.url ~ " + req.url);
                error 200 "ban added";
                }
}


sub vcl_recv {
        if (req.http.host ~"^(www.)?westos.org") {
                set req.http.host = "www.westos.org";
                set req.backend = lb;
                #return (pass);     ##Turn on caching
}elsif (req.http.host ~"^bbs.westos.org"){
                set req.backend = web2;
}else {
                error 404 "error";
}
}
[root@server1 html]# /etc/init.d/varnish reload

Push:

(1) Open the browser and access port 172.25.24.1:8080
firefox
172.25.24.1:8080

(2) Select HTTP

(3) Input and submit the push content. The push is successful:

/index.html

Test: Manual refresh of cache via CDN push
(1) On the test machine:

##Accessing www.westos.org/index.html within a certain period of time because the caching function is turned on, all the content is from the cache.
[root@foundation24 varnish]# curl www.westos.org/index.html
server2:www.westos.org

(2) Re-push/index.html on server server 1
(3)Visit www.westos.org/index.html on the test machine

##Because the cache is refreshed by manual push, access to other back-end servers is re-entered.
[root@foundation24 varnish]# curl www.westos.org/index.html
server3:www.westos.org

Tags: Operation & Maintenance vim curl PHP yum

Posted on Sun, 19 May 2019 00:58:08 -0700 by Cal