अंतरिक्ष विभाग (Department of Space) | भारत सरकार (Government of India)
हिंदी | | | | | | | A+ | A | A- |
prl logo
prl logo

Arch Linux Raspberry Pi Beowulf Cluster

by Samuel Johnson, Girish Padia & Jigar Raval

Introduction

Raspberry Pi is a small (credit card sized), inexpensive single-board computer that is capable of running Linux and other lightweight operating systems which run on ARM processors.

The RPi Cluster project was started a couple weeks ago in response to a PRL's NSD and outreach related activities. We wanted to showcase how simple it is to build a Beowulf cluster to students and in turn explain the architecure of PRL's 100 TF HPC Cluster - Vikram-100.

A Beowulf cluster is simply a collection of identical, (typically) commodity computer hardware based systems, networked together and running some kind of parallel processing software that allows each node in the cluster to share data and computation. Typically, the parallel programming software is MPI (Message Passing Interface), which utilizes TCP/IP along with some libraries to allow programmers to create parallel programs that can split a task into parts suitable to run on multiple machines simultaneously. MPI provides an API that enables both asynchronous and synchronous process interaction.

Building a RPi cluster has few advantages: 1) You will have full root level access to the cluster 2) RPi provides low-level external hardware access (GPIO, UART, etc) 3) RPis are cheap. For the price of a single (node) computer, 20+ RPi compute nodes can be bought. 4) The exact same program running on RPi can be ported to any supercomputing facility and run there with increased performance.

There are several operating systems available as pre-configured Linux images for the RPi: Raspbian (based on Debian) and Arch Linux for ARM (aka “alarm”).

So, which Linux distribution makes sense for a cluster of RPis? This depends greatly on how comfortable you are with Linux. Raspbian provides a turnkey solution in that it comes pre-installed with the LXDE (a full desktop environment), and being Debian based, there is a huge set of packages precompiled and available for the RPi. This is great for getting started with Linux and with the RPi in general. The downside to this distribution is its weight. With support for so many features to begin with, there is a staggering amount of daemons running all the time. As such, boot time is much longer than it has to be, and you have many packages installed that you most likely will never need or use.

Arch LInux on the other hand, takes the minimalist approach. The image is tiny at ~150MB. It boots in around 10 seconds. The install image has nothing extra included. The default installation provides a bare bones, minimal environment, that boots to a command line interface (CLI) with network support. The beauty of this approach is that you can start with the cleanest, fastest setup and only add the things you need for your application. The downside is you have to be willing to wade through the learning process of a different (but elegant) approach to Linux, which is found in Arch Linux. Fortunately, Arch Linux has some of the best organized information out there for learning Linux. An excellent place to start is the Arch Linux Beginner’s Guide.

However, PRL has already customised a version of Arch Linux for RPi which can be installed by following the steps mentioned below:

Prerequisite:

A computer with any variant of Linux either installed or booted from Live CD/DVD/USB. MicroSD card & reader Ethernet cables and network switch Raspberry Pi(s) A lot of will, determination, patience and curiosity.

With all the prerequisites checked, lets get started:

  1. Download the compressed OS disk image
  2. Decompress the downloaded image
    3. $ tar -xvf compute.tar.gz
  3. Attach a Raspberry Pi’s MicroSD card to your computer. After attaching, carefully find the name of the MicroSD Block Device file using the command lsblk.
    4. $ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda           8:0    0 232.9G  0 disk 
├─sda1        8:1    0   256M  0 part /boot
├─sda2        8:2    0   100G  0 part /
├─sda3        8:3    0   100G  0 part 
├─sda4        8:4    0     1K  0 part 
└─sda5        8:5    0  32.6G  0 part /home
mmcblk0     179:0    0   7.4G  0 disk 
├─mmcblk0p1 179:1    0   100M  0 part 
└─mmcblk0p2 179:2    0   7.3G  0 part
  4. Transfer the contents of the decompressed downloaded image to your MicroSD card. Make sure to umount the card if it got automatically/accidentally mounted.
    5. # umount /dev/mmcblk0* 
# dd bs=1m if=/path/to/compute.img of=/dev/mmcblk0
    BE VERY CAREFUL WITH THE ABOVE COMMAND. VERIFY THRICE THAT THE OUTPUT DEVICE IS INDEED YOUR MICROSD CARD. OTHERWISE THIS STEP COULD POTENTIALLY WIPE OUT YOUR PC’S HARD DISK CONTENTS.
  5. Once the image is successfully written, mount the partition in your MicroSD card to your local computer. In most cases, it will be done automatically if you remove and reinsert the card. Otherwise:
    6. # mkdir /mnt/microsd
# mount /dev/mmcblk0p2 /mnt/microsd
  6. Set your hostname (e.g. master) in /mnt/microsd/etc/hostname and IP address (e.g. ip_address) in /mnt/microsd/etc/systemd/network/eth0.network file.
  7. Follow the steps 1 to 5 for a different SD card for another Raspberry Pi and set a different hostname (e.g. compute1) and IP address (e.g. ip_address).
  8. Connect Raspberry Pi(s) and your computer to the network switch.

    9. Logging into the cluster:

  9. Log in to your master Raspberry Pi with alarm as password as
    10. $ ssh alarm@ip_address
  10. Add IP address and hostname of compute nodes to /etc/hosts file in master node
    11. # echo -e 'ip_address\tcompute1' >> /etc/hosts
# echo -e 'ip_address\tcompute2' >> /etc/hosts
...
    Now, you can refer to other compute nodes using their name instead of their IP address.
  11. Generate ssh keys and setup password-less login among master and compute nodes
    12. $ ssh-keygen
$ ssh-copy-id alarm@compute1
$ ssh-copy-id alarm@compute2
...
  12. In the home directory of alarm user there is a directory MPI which contains example MPI code to get started. In the same directory, there is a host file that contains list of nodes where computation can be carried out. You may wish to edit that file depending on the number of Raspberry Pis' you have.
    13. $ cat host 
master slots=4
compute1 slots=4
compute2 slots=4
compute3 slots=4
  13. Compile the sample code as
    14. $ mpiCC mpi_prime.c -o mpi_prime.o
  14. Transfer the compiled code to all nodes (Alternatively NFS can also be configured).
    15. $ for i in `seq 1 3`; do scp * alarm@compute$i:~/MPI/; done;
  15. Run the MPI parallel program
    16. $ mpiexec -n 16 -hostfile host /home/alarm/MPI/mpi_prime.o

    Congratulations! Your Arch Linux Raspberry Pi Beowulf Cluster is Now Ready.

Reference:

  1. http://archlinuxarm.org/platforms/armv7/broadcom/raspberry-pi-2
  2. http://coen.boisestate.edu/ece/files/2013/05/Creating.a.Raspberry.Pi-Based.Beowulf.Cluster_v2.pdf
  3. http://thenewstack.io/installing-mpi-python-raspberry-pi-cluster-runs-docker/
  4. https://apparatusd.wordpress.com/2012/04/18/hpc-high-performance-compute-cluster-with-mpi-and-arch
  5. https://en.wikipedia.org/wiki/Raspberry_Pi