User Tools

Site Tools


Rootfs on External Storage (extroot)

  • More often than not, a limited amount of storage is available on embedded devices. While flash memory will generally accommodate a bare OpenWrt installation, more storage for applications and data can expand a device's potential; with many devices having expansion capabilities built-in (i.e. USB, SATA, and PCIe ports, NAS, etc.).
  • Applications are developed with the idea they should be installed in the root file system (rootfs). By employing extroot, expansion of the storage capacity of your root file system is accomplished by using an added storage device. During the boot process, external storage space is mounted as the root file system, or in an overlay configuration over the original file system. To understand the technical details, please read extroot theory.


  • Configuring the bootloader to boot from an external USB drive, of which most can not.
  • Configuring the bootloader to boot over the network via netboot, of which most can not.
  • Make opkg install new packages somewhere else; however:
    • Installing packages in root is more convenient, as files are installed in locations the OS expects them to reside.
  • Use kexec; however, this requires custom configurations on different devices.

Automatic Setup

  • There's a project called OpenWrt Auto extroot that can build a customized firmware image (using the Image Generator).
    • When this image is flashed, it will automatically set up extroot on any inserted storage device and then reboot (hopefully into the freshly set up extroot).
    • It's a nice foundation for custom applications that require auto provisioning, or just for your convenience at home for an easy extroot setup.
  • This is an ImageBuilder example for building very small images (compatible with 4 MB devices) with only the required packages for manual extroot setup. Please mind that there is no LuCI installed. Once extroot is performed, you can install LuCI and all the packages you require on top of the newly created filesystem.
    make image PROFILE=TLMR3020 PACKAGES="blkid block-mount kmod-fs-ext4 kmod-usb2 kmod-usb-uhci kmod-usb-ohci kmod-usb-storage"


You need to mount a file system, on which the overlay will be copied to.

  • For USB support, follow usb.essentials
  • For USB storage support, follow See storage for general information on partitioning, formatting and mounting storage devices.
  • Alternatively, follow Mounting Filesystems to mount a remote network file system

Make sure that you can mount and have read/write access to your external storage device. For example, check if you can read/write from a manually mounted file system on your partitioned USB drive at /dev/sda (adapt sda to your actual needs):

  mount /dev/sda1 /mnt/sda1

You should check if the mount was succesful by running mount or df. Now check if you can write to /mnt/sda1 by writing a file to it and reading it back.

Chaos Calmer

As described in extroot theory you can use pivot-overlay or pivot-root. This section will only cover pivot-overlay which is the recommended extroot implementation. If you really want pivot-root, check the Barrier Breaker section.


  • block-mount
  • kmod-fs-ext4 or kmod-fs-[filesystem of choice]
  • kmod-usb-storage-extras
    opkg update ; opkg install block-mount kmod-fs-ext4 kmod-usb-storage-extras
  • If opkg errors when installing kmod-usb-storage-extras, you might need to install


  1. Format and partition drive the way you want, either using a Linux Live CD or via uci (it's recommended to use first partition for the root overlay, second partition for swap (if you want swap), and then you can add other partitions as you like)
  2. Prepare your external storage root overlay
    mount /dev/sda1 /mnt ; tar -C /overlay -cvf - . | tar -C /mnt -xf - ; umount /mnt
  3. Create an fstab template
    block detect > /etc/config/fstab ; vi /etc/config/fstab
  4. Edit the following in /etc/config/fstab:
    1. Set all enabled '0' enabled '1'
    2. Edit option target of your root overlay partition to show /overlay (not overlay, as it will cause broken, duplicate mounts)
    3. Edit option target of your other partition(s) to show whatever mount point(s) you want them to have (make sure to also create the directories those mount points point to)
      You'll end up with an fstab looking something like this:
      config 'global'
              option  anon_swap       '0'
              option  anon_mount      '0'
              option  auto_swap       '1'
              option  auto_mount      '1'
              option  delay_root      '5'
              option  check_fs        '0'
      config 'mount'
              option  target  '/overlay'
              option  uuid    'c91232a0-c50a-4eae-adb9-14b4d3ce3de1'
              option  enabled '1'
      config 'swap'
              option  uuid    '08b4f0a3-f7ab-4ee1-bde9-55fc2481f355'
              option  enabled '1'
      config 'mount'
              option  target  '/data'
              option  uuid    'c1068d91-863b-42e2-bcb2-b35a241b0fe2'
              option  enabled '1'
  5. Reboot the router
  6. Verify that the partitions were mounted properly:
    • via LuCI
      • System - Software should show free space of overlay partition
      • System - Mount Points should show USB partition mounted as overlay
    • via CLI
      • mount should show USB partition mounted as /overlay
        /dev/root           on /rom               type squashfs (ro,relatime)
        proc                on /proc              type proc     (rw,noatime)
        sysfs               on /sys               type sysfs    (rw,noatime)
        tmpfs               on /tmp               type tmpfs    (rw,nosuid,nodev,noatime)
        /dev/ubi0_1         on /overlay           type ubifs    (rw,noatime)
        overlayfs:/overlay  on /                  type overlay  (rw,noatime,lowerdir=/,upperdir=/overlay/upper,workdir=/overlay/work)
        tmpfs               on /dev               type tmpfs    (rw,relatime,size=512k,mode=755)
        devpts              on /dev/pts           type devpts   (rw,relatime,mode=600)
        /dev/sda1           on /overlay           type ext4     (rw,relatime,data=ordered)
        /dev/sda3           on /data              type ext4     (rw,relatime,data=ordered)
        debugfs             on /sys/kernel/debug  type debugfs  (rw,noatime)
      • df should show free space available on your /overlay and /data partition
        Filesystem           1K-blocks      Used Available Use% Mounted on
        rootfs                   27180      7796     17964  30% /
        /dev/root                 2304      2304         0 100% /rom
        tmpfs                   127668      1468    126200   1% /tmp
        /dev/ubi0_1            1998672      8056   1869376   0% /overlay
        overlayfs:/overlay       27180      7796     17964  30% /
        tmpfs                      512         0       512   0% /dev
        /dev/sda1              1998672      8056   1869376   0% /overlay
        /dev/sda3            242846048    163864 230323224   0% /data
  7. You're done!


  • Add option force_space in /etc/opkg.conf to allow installation of packets bigger than your /rom partitions free space:
    echo option force_space >> /etc/opkg.conf
  • Do not use vfat (FAT/FAT32); it does not work. If you have a FAT preformatted USB drive, you cannot use it for extroot without reformatting. Use e.g. ext4 (install e2fsprogs, then format your FAT formatted USB drive using mkfs.ext4 /dev/sda1 as per the example, also see storage).
  • If the partition containing your extroot isn't mounted during boot, but you can mount it without problems from a shell, you should try to increase config 'global' / option delay_root. On my system I had to set it to 15 seconds to get extroot working. Another hint to this being the culprit is having a working swap or other partitions mounted after booting, but not your extroot.
  • Another possibility to consider and try is to include in /etc/rc.local the commands: export PREINIT = 1;mount_root, as described in 14946 ticket, which in the case of running Chaos Calmer r44266 in the Comtrend AR-5387un, has been the only thing that allowed me to achieve extroot.


Using Image Generator

As the kmod-fs-ext4 package is rather large, it can be a challenge setting up extroot on 4MB flash devices, where extroot is most needed. Therefore, you can build an image to flash to your device with the required packages already included, this method saves considerable space on your device.

Using the image generator makes building the image very easy. See the Image Generator page for more detailed instructions.

  1. Extract archive, change to directory
  2. To build your firmware (Note: Replace TLMR3220 with the profile for your own device, use make info for a list):
    make image PROFILE="TLMR3220" PACKAGES="kmod-fs-ext4 kmod-usb-storage"

Your new firmware file will be located within the bin folder, with the packages to enable extroot already included.

Remote File Systems

System Upgrade

I recommend that you DO NOT try to do upgrades using opkg upgrade. You will likely end up with an inconsistent state and bricked router that way:

  • The main reason is that the uClibc ABI (Application Binary Interface) is unstable and changes from revision to revision, so binaries for one version of uClibc may be incompatible with versions from another.
  • Another problem that can arise is if you try to upgrade the kernel packages, then flash and reboot, but your operation is interrupted in any way, then you will have a kernel and module mismatch and likely a brick.
  • Finally, if you upgrade all packages but the kernel and the kernel modules, some packages like iptables will be broken.

Accessing original root

Sometimes you may need to access the original root overlay, for example to change your extroot settings. A convenient way of doing this is to configure /etc/config/fstab on your extroot partition to mount the original root overlay in another directory, like this:

config mount
	option target	/overlay-boot
	option device	/dev/mtdblock3
	option fstype	jffs2
	option options	rw,sync
	option enabled	1
	option enabled_fsck 0
This assumes the original, internal overlay was on /dev/mtdblock3 - check your router's page on this wiki and look at the flash map to confirm what MTD block it is for you. Or run $ cat /proc/mtd and search the partition named rootfs_data to know where is your overlay.

If you then create /overlay-boot on the extroot partition, this directory will contain the original root overlay, which is used as the main root overlay until the extroot is up and running. So you can then for example edit /overlay-boot/etc/config/fstab to change your extroot configuration (or temporarily disable it) should you ever need to.

Information on Legacy versions

For extroot on OpenWrt versions Barrier Breaker, Attitude Adjustment, Backfire see: extroot.old

doc/howto/extroot.txt · Last modified: 2016/02/08 14:05 by rhombus