18/06/2018 |

How to Add a Load Balancer in Front of OpenIO SDS

Last week I wrote an article about installing OpenIO SDS, and now I'd like to go a step further by showing you how to add a load balancer in front of it. For the experiments in my home lab I use Odroid SBCs, but you can do the same with Raspberry Pis or x86 machines.

Why Load Balancers?

Before explaining how to configure the load balancer and make it work with OpenIO SDS, I'd like to explain why you need a load balancer.

In a scale-out infrastructure, load balancers spread I/O operations across multiple nodes and help to improve infrastructure availability. OpenIO SDS is a scale-out object store, and all data and I/O operations are distributed across the cluster backend. But, as is usually the case with this kind of architecture, the single front-end I/O operation is managed by a single node in the cluster. In practice, your application knows an IP address and sends requests just to that single address. Without a load balancer all traffic would be directed to a single node, creating a bottleneck.Guillaume, head of pre-sales, explains how OpenIo SDS works

The role of the load balancer is to publish a virtual interface (IP address) and redirect traffic to all the front-end nodes in the cluster. There are several ways to manage traffic redirection and the way you chose to do so depends on the characteristics of your infrastructure and applications. In the case of OpenIO SDS, all the front-end services are stateless, meaning that the nodes do not retain any status or connection information about the communication between the client and the node itself. This simplifies things, and the load balancer can be configured with simple policies like  round-robin.

For load balancing, we usually recommend HAProxy, a fairly common, feature-rich open-source solution.

How to Configure HAProxy for OpenIO SDS

Configuring HAProxy is simple. For this tutorial I am assuming that the load balancer and cluster nodes are on the same network, and am configuring HAProxy for round robin, which is the most common configuration in production for OpenIO SDS.

 

LB-scheme

 

 

The load balancer has a physical interface with an IP address in the 192.169.1.0 network adn its virtual public IP address will be 192.168.1.100, while the cluster nodes will be 192.168.1.1-3.

First, install the HAProxy package on the host you want to use as the load balancer. In Ubuntu it will be:

#apt-get install haproxy

 

Then add a virtual IP address to the server that will host the balancer. To do that you can just add these lines to /etc/network/interface: 

auto eth0:1

iface eth0:1 inet static

    address 192.168.1.100

    netmask 255.255.255.0

 

Bring the virtual interface up with the command:

#Ifconfig eth0:1 192.168.1.100 up

 

Create a /etc/haproxy/haproxy.cfg file with the following lines:

    global
log 127.0.0.1 local0 debug

    chroot /var/lib/haproxy


    nbproc 4


    user haproxy

    group haproxy

    daemon

 

    stats socket /run/haproxy/haproxy.sock mode 0660 level admin

    stats timeout 30

    # SSL

    ca-base /etc/ssl/certs

    crt-base /etc/ssl/private

    # Default ciphers to use on SSL-enabled listening sockets.

    # For more information, see ciphers(1SSL). This list is from:

    #  https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/

    ssl-default-bind-ciphers ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS

    ssl-default-bind-options no-sslv3 no-tlsv10 no-tls-tickets

    ssl-default-server-ciphers ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS

    ssl-default-server-options no-sslv3 no-tlsv10 no-tls-tickets

    tune.ssl.default-dh-param 2048

 

  defaults

    log global

    unique-id-format %{+X}o\ %ci:%cp_%fi:%fp_%Ts_%rt:%pid

    unique-id-header X-Unique-ID

    mode http

    option  httplog

    option  dontlognull

    option  forwardfor

    option  log-separate-errors

    timeout connect 5s

    timeout client  60s

    timeout server  60s

    errorfile 400 /usr/share/haproxy/400.http

    errorfile 403 /usr/share/haproxy/403.http

    errorfile 408 /usr/share/haproxy/408.http

    errorfile 500 /usr/share/haproxy/500.http

    errorfile 502 /usr/share/haproxy/502.http

    errorfile 503 /usr/share/haproxy/503.http

    errorfile 504 /usr/share/haproxy/504.http



  ### OpenIO Swift proxy

  # Public Access

  frontend swift-public

    bind 192.168.1.100:6007

    log-format %ci:%cp\ [%t]\ %ft\ %b/%s\ w=%Tw/c=%Tc/r=%Tr/t=%Tt\ %ST\ %B\ %CC\ %CS\ %tsc\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %hr\ %hs\ %{+Q}r\ %ID

    capture request header Host len 60

    capture request header X-Unique-ID len 46

    capture request header X-Client-IP len 24

    reqadd X-Forwarded-Proto:\ http

    default_backend swift-backend



  backend swift-backend

    balance roundrobin

    server swift-proxy1 192.168.1.1:6007 check inter 5s

    server swift-proxy2 192.168.1.2:6007 check inter 5s

    server swift-proxy3 192.168.1.3:6007 check inter 5s



  ### Conscience

  frontend conscience

    mode tcp

    bind 192.168.1.100:6000 name conscience

    log-format %ci:%cp\ [%t]\ %ft\ %b/%s\ w=%Tw/c=%Tc/t=%Tt\ %B\ %ts\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq

    default_backend conscience-backend



  backend conscience-backend

    mode tcp

    option tcp-check

    server conscience1 192.168.1.1:6000 check inter 5s

 

The second part of the file is the most important. This is the section where all the front-end and back-end services are mapped. Each sub-section starts with frontend <name-of-the-service> followed by backend <name-of-the-service>.  As you can see, the configuration file is pretty simple.

Check your configuration with this command:

#haproxy -c -f /etc/haproxy/haproxy.cfg

 

And start the load balancer with this one:

#systemctl start haproxy
 

Point your application to the virtual IP (192.168.1.100 in our case) and you'll see the traffic pass through the load balancer and be equally distributed to the cluster nodes.

Key Takeaways

Most Linux sysadmins have experience with HAProxy and this short article doesn't tell them anything new. But I had to ask how to configure it when I started working more on OpenIO SDS, and I thought that it could be useful for people who want to try our software and don’t yet have a lot of experience with scale-out infrastructures.

Once again, I have shown that OpenIO SDS is within anyone's reach and can be deployed quickly for testing and - why not? - something more serious than that without issues or particular limitations

If you have any ideas and want to contribute similar articles, please let us know, join our community and get rewarded with our Raspberry Pi Kit!

Watch Guillaume, head of pre-sales, explaining how OpenIo SDS works

COMMENTS