Using external drive as rootfs for raspberry pi

Raspberry pi uses SD/TF card as storage. However, sometimes it becomes a bottleneck of performance because of the I/O speed on SD card is quite limited.

A simple fio read/write test on my setup shows even with USB2.0, an external SSD is way faster than SD card.

sdcard: (groupid=0, jobs=1): err= 0: pid=582: Sun Mar 27 13:36:30 2016
  read : io=65392KB, bw=4442.4KB/s, iops=1110, runt= 14720msec
    clat (usec): min=7, max=18574, avg=175.16, stdev=1657.17
     lat (usec): min=8, max=18575, avg=176.57, stdev=1657.16
    clat percentiles (usec):
     |  1.00th=[    7],  5.00th=[    8], 10.00th=[    8], 20.00th=[    9],
     | 30.00th=[    9], 40.00th=[   10], 50.00th=[   10], 60.00th=[   10],
     | 70.00th=[   11], 80.00th=[   11], 90.00th=[   12], 95.00th=[   15],
     | 99.00th=[  322], 99.50th=[17280], 99.90th=[18048], 99.95th=[18048],
     | 99.99th=[18560]
    bw (KB  /s): min=   93, max=12760, per=95.99%, avg=8527.36, stdev=5281.16
  write: io=65680KB, bw=4461.1KB/s, iops=1115, runt= 14720msec
    clat (usec): min=23, max=3373.8K, avg=699.49, stdev=38712.68
     lat (usec): min=24, max=3373.8K, avg=701.24, stdev=38712.68
    clat percentiles (usec):
     |  1.00th=[   23],  5.00th=[   23], 10.00th=[   24], 20.00th=[   24],
     | 30.00th=[   24], 40.00th=[   24], 50.00th=[   25], 60.00th=[   25],
     | 70.00th=[   26], 80.00th=[   30], 90.00th=[   32], 95.00th=[   45],
     | 99.00th=[   82], 99.50th=[18048], 99.90th=[19840], 99.95th=[20608],
     | 99.99th=[2572288]
    bw (KB  /s): min=   99, max=12832, per=96.08%, avg=8573.43, stdev=5318.31
    lat (usec) : 10=18.57%, 20=29.59%, 50=49.06%, 100=1.59%, 250=0.10%
    lat (usec) : 500=0.30%
    lat (msec) : 4=0.01%, 10=0.01%, 20=0.74%, 50=0.04%, 1000=0.01%
    lat (msec) : >=2000=0.01%
  cpu          : usr=2.38%, sys=4.69%, ctx=1521, majf=0, minf=22
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=16348/w=16420/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
ssd: (groupid=0, jobs=1): err= 0: pid=583: Sun Mar 27 13:36:30 2016
  read : io=65392KB, bw=28543KB/s, iops=7135, runt=  2291msec
    clat (usec): min=7, max=196418, avg=63.48, stdev=1925.74
     lat (usec): min=8, max=196419, avg=64.85, stdev=1925.74
    clat percentiles (usec):
     |  1.00th=[    7],  5.00th=[    8], 10.00th=[    8], 20.00th=[    8],
     | 30.00th=[    9], 40.00th=[    9], 50.00th=[   10], 60.00th=[   10],
     | 70.00th=[   10], 80.00th=[   11], 90.00th=[   12], 95.00th=[   15],
     | 99.00th=[ 1608], 99.50th=[ 1832], 99.90th=[ 2128], 99.95th=[ 2256],
     | 99.99th=[146432]
    bw (KB  /s): min= 6144, max=39320, per=100.00%, avg=27607.50, stdev=15537.13
  write: io=65680KB, bw=28669KB/s, iops=7167, runt=  2291msec
    clat (usec): min=15, max=280132, avg=54.86, stdev=2193.37
     lat (usec): min=17, max=280134, avg=56.55, stdev=2193.37
    clat percentiles (usec):
     |  1.00th=[   16],  5.00th=[   16], 10.00th=[   16], 20.00th=[   17],
     | 30.00th=[   17], 40.00th=[   17], 50.00th=[   17], 60.00th=[   17],
     | 70.00th=[   18], 80.00th=[   18], 90.00th=[   22], 95.00th=[   26],
     | 99.00th=[ 1256], 99.50th=[ 1720], 99.90th=[ 2096], 99.95th=[ 2288],
     | 99.99th=[ 3024]
    bw (KB  /s): min= 6176, max=39392, per=100.00%, avg=27584.75, stdev=15473.86
    lat (usec) : 10=21.33%, 20=70.89%, 50=5.47%, 100=0.41%, 250=0.18%
    lat (usec) : 500=0.14%, 750=0.01%, 1000=0.03%
    lat (msec) : 2=1.33%, 4=0.20%, 250=0.01%, 500=0.01%
  cpu          : usr=15.28%, sys=23.14%, ctx=686, majf=0, minf=22
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=16348/w=16420/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=1

So if you want a better pi, it's highly recommended to use an external disk as rootfs.

Last year, I backed piDrive on Kickstart. The project was aimed to provide a SSD adapter for pi with low power requirement. This board made it possible to use an SSD with Pi. And we can put our rootfs on this disk to run the Pi smoothly.

Before we get started, I urge you to ensure your power adapter could provide 5V/2A for Pi2 and 5V/2.5A for Pi3. Otherwise, you may be resulted in file system corruption. Even worse, the SSD could be destroyed completely, like my first one.

You will still need a SD card as /boot your pi because that's what pi requires. A ~128MB card will be just ok for /boot but I know it's already difficult to find on the market. Perhaps you can create two partitions on your card so you can utilize the remaining spaces.

Create partition on your SSD and format them as you want. You can use ext4 for your rootfs. But for sdcard, only vfat is allowed for the /boot partition. Mount the SSD and sdcard to your current system, for example, as /mnt/root/ and /mnt/boot/.

I'm using archlinuxarm as my os, it's clean and compact as archlinux. The base tarball is just 270MB, makes raspbian's 1G+ seems a little bloated.

Download the tarball from archlinuxarm, at the time of writing, both Pi2 and Pi3 uses Pi2 file system.

Extract the tarball to /mnt/root with:

bsdtar -xpf ArchLinuxARM-rpi-2-latest.tar.gz -C /mnt/root

Copy /boot directory to SD card:

cp -r /mnt/root/boot/* /mnt/boot/
mv /mnt/root/boot /mnt/root/boot_bak

Edit /mnt/boot/cmdline.txt to tell Pi where your rootfs is:

root=/dev/sda1 rootdelay=5 rw rootwait console=ttyAMA0,115200 console=tty1 selinux=0 plymouth.enable=0 smsc95xx.turbo_mode=N dwc_otg.lpm_enable=0 kgdboc=ttyAMA0,115200 elevator=noop

Actually you just need to change two key/value pairs:

Change root= to your SSD partition that contains root fs. Typically your disk will be sda1 here. From my experiments, the bootloader doesn't recognize PARTUUID=... as told by adafruit and /dev/disk/by-uuid/....
Add rootdelay=5 to let bootloader to wait for your partition to be loaded.

Then you will need to edit /mnt/root/etc/fstab to tell linux (not pi) about the file system layout:

#
# /etc/fstab: static file system information
#
# <file system> <dir>   <type>  <options>       <dump>  <pass>
/dev/mmcblk0p1  /boot   vfat    defaults        0       0
/dev/sda1       /       ext4    defaults        0       1

So that's simply all you need to get prepared. Run sync on your system to make sure everything is on mounted disks. Umount sdcard and ssd. Set them up on your Pi. Profit!