11/06/2018 |

How Much Time Does it Take to Install an OpenIO SDS cluster?

The short answer is "less than you may think”, but let me proceed step by step. ;)

The Raspberry Pi Program

A few weeks ago we launched our Raspberry Pi community program. The idea is to reward those who share their experience on OpenIO SDS with a small gift. Contributions could be anything: a blog about an integration, a code contribution, a product review, and so on. The goal is to chose the most interesting ones and send the authors a small kit with all the necessary parts to install OpenIO SDS on a single node. We also built a specific disk image and documentation to get it ready quickly.

More than just one node

But, as usually happens with these things, someone wanted to do more. As soon as the first RPis arrived I got requests to install a fully functional cluster on RPis. I was not 100% sure that everything would work as promised, especially because we are talking about small, low-performance devices, and without testing everything thoroughly you could encounter all sorts of nasty issues.

Object Storage on ARM: Cost Effective, Efficient, and Easy!

We have already demonstrated, several times, that we can run our software on very small devices, including the Raspberry Pi Zero, but most of the time these projects are managed in controlled environments; or, like in the case of the recently launched Ambedded Mars200, there are specific optimizations and configurations applied to make sure that everything works as intended.

And even though we have an office full of RPis and other similar SBCs, our engineers know the product deeply and know how to work around potential issues or hardware limitations. I'm not like them, and I wanted to be sure that anybody could build an RPi cluster easily.

Let's build a 3-node cluster

I wanted to be sure that the installation process is bulletproof or, as one of our engineers told me yesterday, Enrico-proof!

And, to make things a bit harder, I didn't have Raspberry Pis at home but a bunch of Odroids (MC1 + HC1 to be precise).

Prerequisites for this installation are:

  • 3 RPis with Ubuntu 16.04.
  • Ansible 2.2 or newer 
  • SSH access to all the nodes (to simplify, I configured them with key authentication, password-less).
  • Static IP addresses on all the nodes.
  • A user with admin privileges (but I did everything from root).
  • Internet access to download the required packages and files.

This first part took 15 minutes to get the SD cards flashed and another 15 minutes for the first boot and basic Linux configuration.

Once you have the 3 nodes on the same network and able to talk with each other, we can start to install and configure OpenIO SDS, and the process is quite simple.

Download and initiate the installation procedure on one of the nodes:

# mkdir rpi_cluster
# wget http://docs.openio.io/17.04/rpi_cluster.tar.gz
# tar xf rpi_cluster.tar.gz

# cd ansible

If the file "requirements.yml" doesn't exist, create it and add:

# This is used to manage Ansible role dependences
# http://docs.ansible.com/galaxy.html#installing-multiple-roles-from-a-file
# Install with: `ansible-galaxy install -p roles -r requirements.yml`
- src: https://github.com/geerlingguy/ansible-role-ntp
  version: master
  name: geerlingguy.ntp
- src: https://github.com/open-io/ansible-role-openio-sds.git
  version: master
- src: https://github.com/open-io/ansible-role-openio-repository.git
  version: master

Run the following command:

#ansible-galaxy install -p roles -r requirements.yml

You should get output similar to this one:

- geerlingguy.ntp is already installed, skipping.
- ansible-role-openio-sds is already installed, skipping.
- ansible-role-openio-repository is already installed, skipping.


Add an “inventory" file with this content:

node1 ansible_host=IP_node_one
node2 ansible_host=IP_node_two
node3 ansible_host=IP_node_three












Then run this command:

#ansible-playbook -i inventory playbooks/site.yml

This command blurts out a lot of stuff about installation and configuration. But after 15 minutes you should have the cluster up and running (it took less with the Odroids, but CPU and IO are faster).

To see if everything is okay, you can run this command:

root@oio01:~# openio --oio-ns OPENIO cluster list


| Type    | Id                | Volume                          | Location | Slots | Up   | Score |


| rdir    | | /var/lib/oio/sds/OPENIO/rdir-1  | odroid   | n/a   | True |   100 |

| rdir    | | /var/lib/oio/sds/OPENIO/rdir-1  | oio02    | n/a   | True |   100 |

| rdir    | | /var/lib/oio/sds/OPENIO/rdir-1  | oio03    | n/a   | True |   100 |

| account | | n/a                             | odroid   | n/a   | True |   100 |

| account | | n/a                             | oio02    | n/a   | True |   100 |

| account | | n/a                             | oio03    | n/a   | True |   100 |

| rawx    | | /var/lib/oio/sds/OPENIO/rawx-1  | oio02    | n/a   | True |    95 |

| rawx    | | /var/lib/oio/sds/OPENIO/rawx-1  | odroid   | n/a   | True |    95 |

| rawx    | | /var/lib/oio/sds/OPENIO/rawx-1  | oio03    | n/a   | True |    95 |

| meta2   | | /var/lib/oio/sds/OPENIO/meta2-1 | odroid   | n/a   | True |    95 |

| meta2   | | /var/lib/oio/sds/OPENIO/meta2-1 | oio03    | n/a   | True |    95 |

| meta2   | | /var/lib/oio/sds/OPENIO/meta2-1 | oio02    | n/a   | True |    95 |

| meta1   | | /var/lib/oio/sds/OPENIO/meta1-1 | odroid   | n/a   | True |    95 |

| meta1   | | /var/lib/oio/sds/OPENIO/meta1-1 | oio03    | n/a   | True |    95 |

| meta1   | | /var/lib/oio/sds/OPENIO/meta1-1 | oio02    | n/a   | True |    95 |

| meta0   | | /var/lib/oio/sds/OPENIO/meta0-1 | oio01    | n/a   | True |   100 |

| meta0   | | /var/lib/oio/sds/OPENIO/meta0-1 | oio03    | n/a   | True |    99 |

| meta0   | | /var/lib/oio/sds/OPENIO/meta0-1 | oio02    | n/a   | True |   100 |



And that's it! Installing a 3-nodes RPi cluster takes 15 minutes (if you do not count the Linux installation, of course ;) ).

Where to go from here

I installed awscli on one of the nodes and ran some simple commands using the S3 API, with the credentials of a demo user. But you can go to our docs.openio.io to learn about the commands necessary to manage the cluster, or install Keystone (identity management component), or just start using it with your applications.

Key takeways

IMG_0734OpenIO SDS rocks! I installed a full working cluster in 15 minutes (30 minutes if I include the OS).

It runs on very small nodes, which is something I repeat all the time, but today I proved that this is something that anybody can do without any specific tuning or optimizations.

It is easy to use. OpenIO SDS is open source, but some components are available only to paying customers (including the WebUI and the deployment scripts). The files and commands you see in this article are not part of the same procedure you use to install production clusters, but it works well and it is easy to set up a working environment pretty quickly.

Next time I'll add a few disks and to expand the cluster (now running on 16GB SD cards only), and then I could try to run COSBench against it.

Join our community on Slack to get in touch, and, if you have a project or want to test OpenIO SDS with your applications, send me a message; you can easily be included in our Raspberry Pi program and get a free RPi Kit!  

Object Storage on ARM: Cost Efficient, Effective, and Easy with OpenIO on Ambedded Mars200