Arch Linux Raspberry Pi Beowulf Cluster
Raspberry Pi is a small (credit card sized), inexpensive single-board computer that is capable of running Linux and other lightweight operating systems which run on ARM processors.
The RPi Cluster project was started a couple weeks ago in response to a PRL's NSD and outreach related activities. We wanted to showcase how simple it is to build a Beowulf cluster to students and in turn explain the architecure of PRL's 100 TF HPC Cluster - Vikram-100.
A Beowulf cluster is simply a collection of identical, (typically) commodity computer hardware based systems, networked together and running some kind of parallel processing software that allows each node in the cluster to share data and computation. Typically, the parallel programming software is MPI (Message Passing Interface), which utilizes TCP/IP along with some libraries to allow programmers to create parallel programs that can split a task into parts suitable to run on multiple machines simultaneously. MPI provides an API that enables both asynchronous and synchronous process interaction.Building a RPi cluster has few advantages: 1) You will have full root level access to the cluster 2) RPi provides low-level external hardware access (GPIO, UART, etc) 3) RPis are cheap. For the price of a single (node) computer, 20+ RPi compute nodes can be bought. 4) The exact same program running on RPi can be ported to any supercomputing facility and run there with increased performance.
There are several operating systems available as pre-configured Linux images for the RPi: Raspbian (based on Debian) and Arch Linux for ARM (aka “alarm”).
So, which Linux distribution makes sense for a cluster of RPis? This depends greatly on how comfortable you are with Linux. Raspbian provides a turnkey solution in that it comes pre-installed with the LXDE (a full desktop environment), and being Debian based, there is a huge set of packages precompiled and available for the RPi. This is great for getting started with Linux and with the RPi in general. The downside to this distribution is its weight. With support for so many features to begin with, there is a staggering amount of daemons running all the time. As such, boot time is much longer than it has to be, and you have many packages installed that you most likely will never need or use.
Arch LInux on the other hand, takes the minimalist approach. The image is tiny at ~150MB. It boots in around 10 seconds. The install image has nothing extra included. The default installation provides a bare bones, minimal environment, that boots to a command line interface (CLI) with network support. The beauty of this approach is that you can start with the cleanest, fastest setup and only add the things you need for your application. The downside is you have to be willing to wade through the learning process of a different (but elegant) approach to Linux, which is found in Arch Linux. Fortunately, Arch Linux has some of the best organized information out there for learning Linux. An excellent place to start is the Arch Linux Beginner’s Guide.
However, PRL has already customised a version of Arch Linux for RPi which can be installed by following the steps mentioned below:
Prerequisite:A computer with any variant of Linux either installed or booted from Live CD/DVD/USB. MicroSD card & reader Ethernet cables and network switch Raspberry Pi(s) A lot of will, determination, patience and curiosity.
With all the prerequisites checked, lets get started:
- Download the compressed OS disk image
- Decompress the downloaded image
- Attach a Raspberry Pi’s MicroSD card to your computer. After attaching, carefully find the name of the MicroSD Block Device file using the command
- Transfer the contents of the decompressed downloaded image to your MicroSD card. Make sure to
umountthe card if it got automatically/accidentally mounted.
- Once the image is successfully written, mount the partition in your MicroSD card to your local computer. In most cases, it will be done automatically if you remove and reinsert the card. Otherwise:
- Set your hostname (e.g.
/mnt/microsd/etc/hostnameand IP address (e.g.
- Follow the steps 1 to 5 for a different SD card for another Raspberry Pi and set a different hostname (e.g.
compute1) and IP address (e.g.
- Connect Raspberry Pi(s) and your computer to the network switch.
- Log in to your
masterRaspberry Pi with
alarmas password as
- Add IP address and hostname of
- Generate ssh keys and setup password-less login among master and compute nodes
- In the home directory of
alarmuser there is a directory
MPIwhich contains example MPI code to get started. In the same directory, there is a
hostfile that contains list of nodes where computation can be carried out. You may wish to edit that file depending on the number of Raspberry Pis' you have.
- Compile the sample code as
- Transfer the compiled code to all nodes (Alternatively NFS can also be configured).
- Run the MPI parallel program
$ tar -xvf compute.tar.gz
$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 232.9G 0 disk ├─sda1 8:1 0 256M 0 part /boot ├─sda2 8:2 0 100G 0 part / ├─sda3 8:3 0 100G 0 part ├─sda4 8:4 0 1K 0 part └─sda5 8:5 0 32.6G 0 part /home mmcblk0 179:0 0 7.4G 0 disk ├─mmcblk0p1 179:1 0 100M 0 part └─mmcblk0p2 179:2 0 7.3G 0 part
# umount /dev/mmcblk0* # dd bs=1m if=/path/to/compute.img of=/dev/mmcblk0BE VERY CAREFUL WITH THE ABOVE COMMAND. VERIFY THRICE THAT THE OUTPUT DEVICE IS INDEED YOUR MICROSD CARD. OTHERWISE THIS STEP COULD POTENTIALLY WIPE OUT YOUR PC’S HARD DISK CONTENTS.
# mkdir /mnt/microsd # mount /dev/mmcblk0p2 /mnt/microsd
Logging into the cluster:
$ ssh email@example.com
# echo -e '192.168.1.51\tcompute1' >> /etc/hosts # echo -e '192.168.1.52\tcompute2' >> /etc/hosts ...Now, you can refer to other compute nodes using their name instead of their IP address.
$ ssh-keygen $ ssh-copy-id alarm@compute1 $ ssh-copy-id alarm@compute2 ...
$ cat host master slots=4 compute1 slots=4 compute2 slots=4 compute3 slots=4
$ mpiCC mpi_prime.c -o mpi_prime.o
$ for i in `seq 1 3`; do scp * alarm@compute$i:~/MPI/; done;
$ mpiexec -n 16 -hostfile host /home/alarm/MPI/mpi_prime.o
Congratulations! Your Arch Linux Raspberry Pi Beowulf Cluster is Now Ready.