Rapidly configure SD cards for your Raspberry Pi cluster

There are many reasons people want to create computer clusters using the Raspberry Pi, including that they have full control over their platform, they're able to use an inexpensive, highly usable platform, and get the opportunity to learn about cluster computing in general.

There are different methods for setting up a cluster, such as headless, network booting, and booting from SD cards. Each method has advantages and disadvantages, but the latter method is most familiar to users who have worked with a single Pi. Most cluster setups involve many complex steps that require a significant amount of time because they are executed on an individual Pi. Even starting is non-trivial, as you need to set up a network to access them.

Despite improvements to the Raspberry Pi Imager and the availability of PiBakery, the process is still too involved. So, at Cloudmesh, we asked:

Is it possible to develop a tool that is specifically targeted to burn SD cards for Pis in a cluster one at a time so that the cards can be just plugged in and, with minimal effort, start a cluster that simply works?

In response, we developed a tool called Cloudmesh Pi Burner for SD Cards, and we present it within Pi Planet. No more spending hours upon hours to replicate the steps and learn complex DevOps tutorials; instead, you can get a cluster set up with just a few commands.

For this, we developed cms burn, which is a program that you can execute on a "manager" Pi or a Linux or macOS computer to burn cards for your cluster.

We set up a comprehensive package on GitHub that can be installed easily. You can read about it in detail in the README. There, you can also find detailed instructions on how to burn directly from a macOS or Linux computer.

Getting started

This article explains how to create a cluster setup using five Raspberry Pi units (you need a minimum of two, but this method also works for larger numbers). To follow along, you must have five SD cards, one for each of the five Pi units. It's helpful to have a network switch (managed or unmanaged) with five Ethernet cables (one for each Pi).

Requirements

You need:

• 5 Raspberry Pi boards
• 5 SD cards
• 5 Ethernet cables
• A network switch (unmanaged or managed)
• WiFi access
• Monitor, mouse, keyboard (for desktop access on Pi)
• An SD card slot for your computer or the manager Pi (and preferably supports USB 3.0 speeds)
• If you're doing this on a Mac, you must install XCode and Homebrew

On Linux, the open source ext4 filesystem is supported by default. However, Apple doesn't provide this capability for macOS, so you must purchase support separately. I use Paragon Software's extFS application. Like macOS itself, this is largely based upon, but is not itself, open source.

At Cloudmesh, we maintain a list of hardware parts you need to consider when setting up a cluster.

Network configuration

Figure 1 shows our network configuration. Of the five Raspberry Pi computers, one is dedicated as a manager and four are workers. Using WiFi for the manager Pi allows you to set it up anywhere in your house or other location (other configurations are discussed in the README).

Our configuration uses an unmanaged network switch, where the manager and workers communicate locally with each other, and the manager provides internet access to the workers over a bridge that's configured for you.

Image by:

^{Pi cluster setup with bridge network (©2021 The Cloudmesh Projects)}

Set up the Cloudmesh burn application

To set up the Cloudmesh burn program, first create a Python venv:

$ python3 -m venv ~/ENV3
$ source ~/ENV3/bin/activate

Next, install the Cloudmesh cluster generation tools and start the burn process. You must adjust the path to your SD card, which differs depending on your system and what kind of SD card reader you're using. Here's an example:

(ENV3)$ pip install cloudmesh-pi-cluster
(ENV3)$ cms help
(ENV3)$ cms burn info 
(ENV3)$ cms burn cluster \
--device=/path/to/sdcard \
--hostname=red,red01,red02,red03,red04 \
--ssid=myssid -y

Fill out the passwords and plug in the SD cards as requested.

Start your cluster and configure it

Plug the burned SD cards into the Pis and switch them on. Execute the ssh command to log into your manager—it's the one called red (worker nodes are identified by number):

(ENV3)$ ssh pi@red.local

This takes a while, as the filesystems on the SD cards need to be installed, and configurations such as Country, SSH, and WiFi need to be activated.

Once you are in the manager, install the Cloudmesh cluster software in it. (You could have done this automatically, but we decided to leave this part of the process up to you to give you maximum flexibility.)

pi@red:~ $ curl -Ls \
http://cloudmesh.github.io/get/pi \
--output install.sh
pi@red:~ $ sh ./install.sh

After lots of log messages, you see:

#################################################
# Install Completed                             #
#################################################
Time to update and upgarde: 339 s
Time to install the venv:   22 s
Time to install cloudmesh:  185 s
Time for total install:     546 s
Time to install: 546 s
#################################################
Please activate with
    source ~/ENV3/bin/activate

Reboot:

pi@red:~ $ sudo reboot

Start using your cluster

Log in to your manager Pi over SSH:

(ENV3)$ ssh pi@red.local

Once you're logged into your manager (in this example, red.local) on the network, execute a command to see if things are working. For example, you can use a temperature monitor to get the temperature from all Pi boards:

(ENV3) pi@red:~ $ cms pi temp red01,red02,red03,red04

pi temp red01,red02
+--------+--------+-------+----------------------------+
| host   |    cpu |   gpu | date                       |
|--------+--------+-------+----------------------------|
| red01  | 45.277 |  45.2 | 2021-02-23 22:13:11.788430 |
| red02  | 42.842 |  42.8 | 2021-02-23 22:13:11.941566 |
| red02  | 43.356 |  42.8 | 2021-02-23 22:13:11.961245 |
| red02  | 44.124 |  42.8 | 2021-02-23 22:13:11.981896 |
+--------+--------+-------+----------------------------+

Access the workers

It's even more convenient to access the workers, so we designed a tunnel command that makes setup easy. Call it on the manager node, for example:

(ENV3) pi@red:~ $ cms host setup "red0[1-4]" user@laptop.local

This creates ssh keys on all workers, gathers ssh keys from all hosts, and scatters the public keys to the manager's and worker's authorized key file. This also makes the manager node a bridge for the worker nodes so they can have internet access. Now our laptop we update our ssh config file with the following command.

(ENV3)$ cms host config proxy pi@red.local red0[1-4] 

Now you can access the workers from your computer. Try it out with the temperature program:

(ENV3)$ cms pi temp "red,red0[1-4]"              

+-------+--------+-------+----------------------------+
| host  |    cpu |   gpu | date                       |
|-------+--------+-------+----------------------------|
| red   | 50.147 |  50.1 | 2021-02-18 21:10:05.942494 |
| red01 | 51.608 |  51.6 | 2021-02-18 21:10:06.153189 |
| red02 | 45.764 |  45.7 | 2021-02-18 21:10:06.163067 |
...
+-------+--------+-------+----------------------------+

More information

Since this uses SSH keys to authenticate between the manager and the workers, you can log directly into the workers from the manager. You can find more details in the README and on Pi Planet. Other Cloudmesh components are discussed in the Cloudmesh manual.

This article is based on Easy Raspberry Pi cluster setup with Cloudmesh from MacOS and is republished with permission.