Configure Varnish Cache as Load Balancer on CentOS 7

Configure Varnish Cache as Load Balancer on CentOS 7

Varnish Cache is an HTTP Accelerator designed for content-heavy dynamic websites and APIs. Varnish Cache is usually installed on the same web server, where Varnish Cache acts as the front-end and accelerator for the hosted websites. Varnish Cache is free and open source software which is distributed under two-clause BSD license.

Varnish cache supports multiple back-end hosts, therefore we can also used Varnish Cache as the Reverse Proxy for load balancing of a cluster of web servers.

In this article, we are installing and configuring Varnish Cache as Load Balancer on CentOS 7.

 

This Article Provides:

     

    Environment Specification:

    We have configured a CentOS 7 virtual machine with following specifications:

    • CPU - 3.4 Ghz (1 Core)
    • Memory - 1 GB
    • Storage - 20 GB
    • Operating System - CentOS 7.7
    • Hostname - varnish-cache-01.example.com
    • IP Address - 192.168.116.213 /24

     

    Installing Apache HTTP Server on CentOS 7:

    Connect with varnish-cache-01.example.com using ssh as root user.

    Build yum cache for standard CentOS 7 repositories.

    [root@varnish-cache-01 ~]# yum makecache fast
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
    * base: mirrors.ges.net.pk
    * extras: mirrors.ges.net.pk
    * updates: mirrors.ges.net.pk
    base | 3.6 kB 00:00
    extras | 2.9 kB 00:00
    updates | 2.9 kB 00:00
    Metadata Cache Created

    Update CentOS 7 server packages.

    [root@varnish-cache-01 ~]# yum update
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
    * base: mirrors.ges.net.pk
    * extras: mirrors.ges.net.pk
    * updates: mirrors.ges.net.pk
    No packages marked for update

    Our CentOS 7 server is already up-to-date.

    Install Apache HTTP server using yum command.

    [root@varnish-cache-01 ~]# yum install -y httpd
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
    * base: mirrors.ges.net.pk
    * extras: mirrors.ges.net.pk
    * updates: mirrors.ges.net.pk
    Resolving Dependencies
    --> Running transaction check
    ---> Package httpd.x86_64 0:2.4.6-90.el7.centos will be installed
    --> Processing Dependency: httpd-tools = 2.4.6-90.el7.centos for package: httpd-2.4.6-90.el7.centos.x86_64
    --> Processing Dependency: /etc/mime.types for package: httpd-2.4.6-90.el7.centos.x86_64
    --> Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-2.4.6-90.el7.centos.x86_64
    --> Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-2.4.6-90.el7.centos.x86_64
    --> Running transaction check
    ---> Package apr.x86_64 0:1.4.8-5.el7 will be installed
    ---> Package apr-util.x86_64 0:1.5.2-6.el7 will be installed
    ---> Package httpd-tools.x86_64 0:2.4.6-90.el7.centos will be installed
    ---> Package mailcap.noarch 0:2.1.41-2.el7 will be installed
    --> Finished Dependency Resolution

    Dependencies Resolved

    ================================================================================
    Package Arch Version Repository Size
    ================================================================================
    Installing:
    httpd x86_64 2.4.6-90.el7.centos base 2.7 M
    Installing for dependencies:
    apr x86_64 1.4.8-5.el7 base 103 k
    apr-util x86_64 1.5.2-6.el7 base 92 k
    httpd-tools x86_64 2.4.6-90.el7.centos base 91 k
    mailcap noarch 2.1.41-2.el7 base 31 k

    Transaction Summary
    ================================================================================
    Install 1 Package (+4 Dependent packages)

    Total download size: 3.0 M
    Installed size: 10 M
    Downloading packages:
    (1/5): mailcap-2.1.41-2.el7.noarch.rpm | 31 kB 00:01
    (2/5): apr-util-1.5.2-6.el7.x86_64.rpm | 92 kB 00:01
    (3/5): httpd-tools-2.4.6-90.el7.centos.x86_64.rpm | 91 kB 00:02
    (4/5): apr-1.4.8-5.el7.x86_64.rpm | 103 kB 00:04
    (5/5): httpd-2.4.6-90.el7.centos.x86_64.rpm | 2.7 MB 00:08
    --------------------------------------------------------------------------------
    Total 379 kB/s | 3.0 MB 00:08
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
    Installing : apr-1.4.8-5.el7.x86_64 1/5
    Installing : apr-util-1.5.2-6.el7.x86_64 2/5
    Installing : httpd-tools-2.4.6-90.el7.centos.x86_64 3/5
    Installing : mailcap-2.1.41-2.el7.noarch 4/5
    Installing : httpd-2.4.6-90.el7.centos.x86_64 5/5
    Verifying : apr-1.4.8-5.el7.x86_64 1/5
    Verifying : mailcap-2.1.41-2.el7.noarch 2/5
    Verifying : httpd-tools-2.4.6-90.el7.centos.x86_64 3/5
    Verifying : apr-util-1.5.2-6.el7.x86_64 4/5
    Verifying : httpd-2.4.6-90.el7.centos.x86_64 5/5

    Installed:
    httpd.x86_64 0:2.4.6-90.el7.centos

    Dependency Installed:
    apr.x86_64 0:1.4.8-5.el7 apr-util.x86_64 0:1.5.2-6.el7
    httpd-tools.x86_64 0:2.4.6-90.el7.centos mailcap.noarch 0:2.1.41-2.el7

    Complete!

    Start and enable Apache web service.

    [root@varnish-cache-01 ~]# systemctl enable --now httpd.service
    Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.

    Allow HTTP service in CentOS 7 firewall.

    [root@varnish-cache-01 ~]# firewall-cmd --permanent --add-service=http
    success
    [root@varnish-cache-01 ~]# firewall-cmd --reload
    success

    Browse URL http://varnish-cache-01.example.com in a client's browser.

    01-varnish-cache-centos-7-apache-default-page

    Apache HTTP server is successfully installed and it is serving the default test page.

     

    Configure Apache Virtual Hosts on CentOS 7:

    We are configure two virtual hosts here, that will run on two different ports.

    Create document root directories for virtual hosts.

    [root@varnish-cache-01 ~]# mkdir /var/www/html/{vhost1,vhost2}

    Create default index page for Virtual Host 1.

    [root@varnish-cache-01 ~]# cat > /var/www/html/vhost1/index.html << EOF
    > <html>
    > <head><title>Virtual Host1</title></head>
    > <body><h1>This is the default page of Virtual Host 1...</h1></body>
    > </html>
    > EOF

    Similarly, create default index page for Virtual Host 2.

    [root@varnish-cache-01 ~]# cat > /var/www/html/vhost2/index.html << EOF
    > <html>
    > <head><title>Virtual Host2</title></head>
    > <body><h1>This is the default page of Virtual Host 2...</h1></body>
    > </html>
    > EOF

    Create configuration file for Virtual Host1.

    [root@varnish-cache-01 ~]# vi /etc/httpd/conf.d/vhost1.conf

    add following directives therein.

    Listen 8081
    <VirtualHost *:8081>
    DocumentRoot "/var/www/html/vhost1"
    ServerName vhost1.example.com
    </VirtualHost>

    Similarly, create configuration file for Virtual Host2.

    [root@varnish-cache-01 ~]# vi /etc/httpd/conf.d/vhost2.conf

    add following directives therein.

    Listen 8082
    <VirtualHost *:8082>
    DocumentRoot "/var/www/html/vhost2"
    ServerName vhost2.example.com
    </VirtualHost>

    Check Apache configurations for syntax errors.

    [root@varnish-cache-01 ~]# httpd -t
    Syntax OK

    Since, we are running Apache websites on non-default ports, therefore, we have to add these ports to SELinux port labeling.

    Check, if these ports are already added in SELinux.

    [root@varnish-cache-01 ~]# semanage port -l | grep ^http_port_t
    http_port_t tcp 80, 81, 443, 488, 8008, 8009, 8443, 9000

    Add ports 8081 and 8082 to type http_port_t SELinux context.

    [root@varnish-cache-01 ~]# semanage port -m -t http_port_t -p tcp 8081
    [root@varnish-cache-01 ~]# semanage port -m -t http_port_t -p tcp 8082

    Verify if these ports are added in SELinux port labeling.

    [root@varnish-cache-01 ~]# semanage port -l | grep ^http_port_t
    http_port_t tcp 8082, 8081, 80, 81, 443, 488, 8008, 8009, 8443, 9000

    Now, we can safely load our Apache configurations.

    [root@varnish-cache-01 ~]# systemctl reload httpd.service

    Allow 8081/tcp and 8082/tcp service ports in CentOS 7 firewall.

    [root@varnish-cache-01 ~]# firewall-cmd --permanent --add-port={8081,8082}/tcp
    success
    [root@varnish-cache-01 ~]# firewall-cmd --reload
    success

    Browse URL http://varnish-cache-01.example.com:8081/ in a client's browser.

    02-varnish-cache-centos-7-apache-virtual-host-1

    Browse URL http://varnish-cache-01.example.com:8082/ in a client's browser.

    03-varnish-cache-centos-7-apache-virtual-host-2

    Both of our Apache virtual hosts has been configured successfully.

     

    Installing Varnish Cache on CentOS 7:

    Varnish Cache software is available in EPEL (Extra Packages for Enterprise Linux) yum repository.

    Therefore, first we have to enable EPEL yum repository as follows.

    [root@varnish-cache-01 ~]# yum install -y epel-release
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
    * base: mirrors.ges.net.pk
    * extras: mirrors.ges.net.pk
    * updates: mirrors.ges.net.pk
    Resolving Dependencies
    --> Running transaction check
    ---> Package epel-release.noarch 0:7-11 will be installed
    --> Finished Dependency Resolution

    Dependencies Resolved

    ================================================================================
    Package Arch Version Repository Size
    ================================================================================
    Installing:
    epel-release noarch 7-11 extras 15 k

    Transaction Summary
    ================================================================================
    Install 1 Package

    Total download size: 15 k
    Installed size: 24 k
    Downloading packages:
    epel-release-7-11.noarch.rpm | 15 kB 00:00
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
    Installing : epel-release-7-11.noarch 1/1
    Verifying : epel-release-7-11.noarch 1/1

    Installed:
    epel-release.noarch 0:7-11

    Complete!

    Build cache for EPEL yum repository.

    [root@varnish-cache-01 ~]# yum makecache
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
    epel/x86_64/metalink | 7.0 kB 00:00
    * base: mirrors.ges.net.pk
    * epel: sg.fedora.ipserverone.com
    * extras: mirrors.ges.net.pk
    * updates: mirrors.ges.net.pk
    base | 3.6 kB 00:00
    epel | 5.3 kB 00:00
    extras | 2.9 kB 00:00
    updates | 2.9 kB 00:00
    (1/13): epel/x86_64/group_gz | 88 kB 00:01
    (2/13): epel/x86_64/prestodelta | 1.6 kB 00:00
    (3/13): epel/x86_64/updateinfo | 1.0 MB 00:24
    (4/13): base/7/x86_64/other_db | 2.6 MB 00:34
    (5/13): base/7/x86_64/filelists_db | 7.3 MB 00:35
    (6/13): extras/7/x86_64/filelists_db | 207 kB 00:01
    (7/13): extras/7/x86_64/other_db | 100 kB 00:01
    (8/13): updates/7/x86_64/filelists_db | 1.4 MB 00:05
    (9/13): epel/x86_64/primary_db | 6.9 MB 00:43
    (10/13): epel/x86_64/other_db | 3.3 MB 00:25
    (11/13): updates/7/x86_64/other_db | 134 kB 00:18
    (12/13): epel/x86_64/filelists_db | 11 MB 01:02
    (13/13): epel/x86_64/updateinfo_zck | 1.5 MB 00:19
    Metadata Cache Created

    Now, we can install Varnish Cache software using yum command.

    [root@varnish-cache-01 ~]# yum install -y varnish
    ...
    Installed:
    varnish.x86_64 0:4.0.5-1.el7

    Dependency Installed:
    cpp.x86_64 0:4.8.5-39.el7
    dwz.x86_64 0:0.11-3.el7
    gcc.x86_64 0:4.8.5-39.el7
    glibc-devel.x86_64 0:2.17-292.el7
    glibc-headers.x86_64 0:2.17-292.el7
    jemalloc.x86_64 0:3.6.0-1.el7
    kernel-headers.x86_64 0:3.10.0-1062.1.2.el7
    libmpc.x86_64 0:1.0.1-3.el7
    mpfr.x86_64 0:3.1.1-4.el7
    perl.x86_64 4:5.16.3-294.el7_6
    perl-Carp.noarch 0:1.26-244.el7
    perl-Encode.x86_64 0:2.51-7.el7
    perl-Exporter.noarch 0:5.68-3.el7
    perl-File-Path.noarch 0:2.09-2.el7
    perl-File-Temp.noarch 0:0.23.01-3.el7
    perl-Filter.x86_64 0:1.49-3.el7
    perl-Getopt-Long.noarch 0:2.40-3.el7
    perl-HTTP-Tiny.noarch 0:0.033-3.el7
    perl-PathTools.x86_64 0:3.40-5.el7
    perl-Pod-Escapes.noarch 1:1.04-294.el7_6
    perl-Pod-Perldoc.noarch 0:3.20-4.el7
    perl-Pod-Simple.noarch 1:3.28-4.el7
    perl-Pod-Usage.noarch 0:1.63-3.el7
    perl-Scalar-List-Utils.x86_64 0:1.27-248.el7
    perl-Socket.x86_64 0:2.010-4.el7
    perl-Storable.x86_64 0:2.45-3.el7
    perl-Text-ParseWords.noarch 0:3.29-4.el7
    perl-Time-HiRes.x86_64 4:1.9725-3.el7
    perl-Time-Local.noarch 0:1.2300-2.el7
    perl-constant.noarch 0:1.27-2.el7
    perl-libs.x86_64 4:5.16.3-294.el7_6
    perl-macros.x86_64 4:5.16.3-294.el7_6
    perl-parent.noarch 1:0.225-244.el7
    perl-podlators.noarch 0:2.5.1-3.el7
    perl-srpm-macros.noarch 0:1-8.el7
    perl-threads.x86_64 0:1.87-4.el7
    perl-threads-shared.x86_64 0:1.43-6.el7
    python-srpm-macros.noarch 0:3-32.el7
    redhat-rpm-config.noarch 0:9.1.0-88.el7.centos
    varnish-libs.x86_64 0:4.0.5-1.el7
    zip.x86_64 0:3.0-11.el7

    Complete!

    We have installed the default version of Varnish Cache that is available in EPEL yum repository. However, you can always download and install a latest version of Varnish Cache from their Official Download Page.

     

    Configure Varnish Cache as Load Balancer on CentOS 7:

    Before configuring Varnish Cache, we are required to free the port 80 that is currently used by Apache HTTP server.

    The directive that controls the service port 80 is defined in /etc/httpd/conf/httpd.conf file.

    We can change it using a sed command.

    [root@varnish-cache-01 ~]# sed -i "s/Listen 80/Listen 8080/" /etc/httpd/conf/httpd.conf

    Restart the Apache service to take changes into effect.

    [root@varnish-cache-01 ~]# systemctl restart httpd.service

    Now, port 80 is available and we can use it for Vanish Cache service.

    Edit Varnish Cache configuration file.

    [root@varnish-cache-01 ~]# vi /etc/varnish/varnish.params

    Locate and set following directive therein.

    VARNISH_LISTEN_PORT=80 #Default Port 6081

    We have changed the Varnish Cache default port 6081 with 80.

    It's time to configure the backend for Varnish Cache server.

    These settings are located in /etc/varnish/default.vcl file. We can easily replace this file with our custom configurations.

    Rename the existing default.vcl file using mv command.

    [root@varnish-cache-01 ~]# mv /etc/varnish/default.vcl /etc/varnish/default.vcl.org

    Create a custom backend configuration file.

    [root@varnish-cache-01 ~]# vi /etc/varnish/default.vcl

    and add following lines of codes.

    vcl 4.0;

    import directors; # Load the directors

    backend vhost1 {
    .host = "192.168.116.213";
    .port = "8081";
    .probe = {
    .url = "/";
    .timeout = 1s;
    .interval = 5s;
    .window = 5;
    .threshold = 3;
    }
    }

    backend vhost2 {
    .host = "192.168.116.213";
    .port = "8082";
    .probe = {
    .url = "/";
    .timeout = 1s;
    .interval = 5s;
    .window = 5;
    .threshold = 3;
    }
    }

    sub vcl_init {
    new lb = directors.round_robin(); # Creating a Load Balancer
    lb.add_backend(vhost1); # Add Virtual Host 1
    lb.add_backend(vhost2); # Add Virtual Host 2
    }

    sub vcl_recv {
    # send all traffic to the lb director:
    set req.backend_hint = lb.backend();
    }

    Enable and start Varnish Cache service.

    [root@varnish-cache-01 ~]# systemctl enable --now varnish.service
    Created symlink from /etc/systemd/system/multi-user.target.wants/varnish.service to /usr/lib/systemd/system/varnish.service.

    Enable and start Varnish Cache logging service.

    [root@varnish-cache-01 ~]# systemctl enable --now varnishlog.service
    Created symlink from /etc/systemd/system/multi-user.target.wants/varnishlog.service to /usr/lib/systemd/system/varnishlog.service.

    Verify the backend list using following command.

    [root@varnish-cache-01 ~]# varnishadm backend.list
    Backend name Refs Admin Probe
    vhost1(192.168.116.213,,8081) 1 probe Healthy 5/5
    vhost2(192.168.116.213,,8082) 1 probe Healthy 5/5

    Check our website’s response header.

    [root@varnish-cache-01 ~]# curl -I http://varnish-cache-01.example.com

    HTTP/1.1 200 OK
    Date: Sun, 13 Oct 2019 16:24:07 GMT
    Server: Apache/2.4.6 (CentOS)
    Last-Modified: Sun, 13 Oct 2019 09:35:58 GMT
    ETag: "7d-594c77a7e0839"
    Content-Length: 125
    Content-Type: text/html; charset=UTF-8
    X-Varnish: 32770
    Age: 0
    Via: 1.1 varnish-v4
    Connection: keep-alive

    Browse URL http://varnish-cache-01.example.com in a client's browser.

    02-varnish-cache-centos-7-apache-virtual-host-1

    03-varnish-cache-centos-7-apache-virtual-host-2

    The Varnish Cache load balancer redirects user requests to Virtual Host 1 and Virtual Host 2 in a round robin way.

    We have successfully configured Varnish Cache as load balancer on CentOS 7.

    0 Comments