Partitioning, Formatting and Mounting Storage Devices

Being able to add storage to an OpenWrt system can be a very valuable option, because of the usually limited amount of flash on embedded devices. Having ample storage for applications and other data can therefore greatly expand OpenWrt functionality. For example you can turn your device into a full NAS by adding a hard drive for storing media files. You can also add storage to increase the available storage for the root file system in an overlay configuration by employing extroot.

Before the system can find your device it has to be properly recognised, which is done by installing the correct drivers for the device. For most devices the connection will be through USB, so you first have to install USB support and then provide storage support over USB. Other devices are possible too, like SATA, IDE and flash memory cards, see sata.essentials, ide.essentials and cf.essentials, respectively. This article will assume that your storage device is already installed correctly with the correct drivers and ready for use.

After installing the drivers for a device, it has to be partitioned and formatted with a File system (e.g. ext4, FAT) before data can be addressed as files and mountedby the OS. The usual Linux tools can be used to achieve these tasks from within OpenWrt. Certain packages are necessary for specific file system support and other packages facilitate more advanced functionality. This article explains the specifics and caveats of working with storage devices in OpenWrt and provides examples.

Partitioning

Partitioning means to write a partition table to the device, which defines which parts of the available space on the device belongs to which partition, each of which can have its own file system. Most commonly used partition tables are the Master Boot Record (MBR) and the GUID Partition Table (GPT). The latter is more modern and supports larger disks and more partitions. A device can also be used without a partition table, in which case there is only one file system on the device.

You can partition your device before connecting it to your OpenWrt powered device (e.g. on a PC), which may provide a wider range of options, like GPT support. You can then also format any partitions before attaching your device. However, there are packages in the OpenWrt repositories with which you can partition your devices:

  • fdisk is the standard tool to partition drives in Linux. for help with partitioning see TLDP)
  • cfdisk is a curses-based program for partitioning drives. Similar to fdisk, but provides a convenient menu based interface.
  • sfdisk is an alternative to list the size of a partition, list the partitions on a device, check the partitions on a device and repartition a device.
All these utilities support MBR based partition tables only. Currently there are no tools in the default OPKG repository to create and partition GPT based devices. The Linux tools parted and gdisk have not yet been included. Note that while it cannot create GPT tables, there is full support for GPT disks in the OpenWrt kernel, so prepartitioned drives with GPT will work. Also note that if you want to erase a GPT partition table and clear the device to start over with a MBR partition table you have to delete the backup copies of the table that GPT defines, usually also at the end of the device. Since there is no true GPT tool for this in the repository yet, you can erase the tables with the dd tool. This is a parted makefile for OpenWrt that someone provides. After adding dependencies for libblkid and libuuid the package builds fine. Maybe someone should send a patch to the devel-mailinglist.

Upon having your device partitioned, its partitions should be available as 'Device files' under /dev/. For example /dev/sda as the device, with /dev/sda1, /dev/sda2, … , the partitions. Or, in case it is not partitioned with a partition table and it has a single file system, it may be directly accessible under e.g. /dev/sda. Subsequent devices you add will be /dev/sdb, /dev/sdc and so on. These 'device files' can now be partitioned (see below).

Formatting Filesystems

Available file systems and their packages can be listed with opkg update; opkg list "kmod-fs*". These are kernel packages that provide necessary kernel support for reading and writing these file systems. The following table lists these file system kernel packages.

Name Description
kmod-fs-autofs4 Kernel module for AutoFS4 support
kmod-fs-btrfs Kernel module for Btrfs support
kmod-fs-cifs Kernel module for CIFS support
kmod-fs-exportfs Kernel module for exportfs. Needed for some other modules.
kmod-fs-ext4 Kernel module for ext2, ext3 and ext4 filesystem support1)
kmod-fs-hfs Kernel module for HFS filesystem support
kmod-fs-hfsplus Kernel module for HFS+ filesystem support
kmod-fs-isofs Kernel module for ISO9660 filesystem support
kmod-fs-minix Kernel module for MINIX file system support
kmod-fs-msdos Kernel module for MSDOS filesystem support
kmod-fs-nfs Kernel module for NFS support
kmod-fs-nfs-common Common NFS filesystem modules
kmod-fs-nfsd Kernel module for NFS kernel server support
kmod-fs-ntfs Kernel module for NTFS read only filesystem support
kmod-fs-reiserfs Kernel module for ReiserFS support
kmod-fs-udf Kernel module for UDF filesystem support
kmod-fs-vfat Kernel module for VFAT filesystem support
kmod-fs-xfs Kernel module for XFS support

The choice of file system is dependent on several factors: performance, compatibility with other systems, functionality such as compression and journaling, security, etc. The remarks below are important to keep in mind when choosing your file system.

OpenWrt-Forum: Benchmarks with different filesystems

We recommend the file systems ext2, ext3, or ext4 (all supported by kmod-fs-ext4)
As of r26645 the modules for ext2 and ext3 (kmod-fs-ext2 and kmod-fs-ext3) are no longer available. You can use the module for ext4 kmod-fs-ext4 to mount ext2/ext3 as well, but then you have to mount your ext2 or ext3 partition like ext4 partition: mount -t ext4 /dev/sda1 /mnt/share -o rw,sync
If you are planning on sharing your USB storage with Samba, and/or using it to share large multimedia content over UPnP, e.g. with ushare, you should use ext3 or ext4 rather than ext2. Default Ext2 partitions will be limited to files of no more than 4 Gb, but this restriction is eliminated with the newer filesystems. To retain good performance with the newer filesystems, particularly on flash USB storage, you should also consider disabling journaling with the "-O ^has_journal" when using mkfs.ext4 to format the filesystem. Also for ext4, the "extent" formatting option (e.g. "-O ^has_journal,extent") will make filesystem access to large files more performant.
With Ext2Fsd you obtain read/write access to ext2 ext3 and ext4 filesystems in Microsoft Windows. Native Linux file systems like ext2 or ext3 are the first choice when working with Linux. Don't bother with NTFS, VFAT and the like. They all have some quirks we could do without.
We do NOT recommend using NTFS. While it is possible, it has serious performance issues under Linux.
If, for whatever reasons, you insist to utilize NTFS as file system, you may find Writable NTFS helpful.

Available NLS files

For some filesystems, like for VFAT, you need to additionally install language packages (codepages / charsets) to handle the filenames. If your mount fails, look in dmesg - a message like

FAT: codepage cp437 not found

means that you need NLS codepage 437, and a message like

FAT: IO charset iso8859-1 not found

means that you need NLS ISO 8859-1.

***''kmod-nls-base''** Kernel module for NLS (Native Language Support)

Available NLS files and their packages can be listed with opkg update; opkg list "kmod-nls*".

Name Description
kmod-nls-cp1250 Kernel module for NLS Codepage 1250 (Eastern Europe)
kmod-nls-cp1251 Kernel module for NLS Codepage 1251 (Russian)
kmod-nls-cp437 Kernel module for NLS Codepage 437 (United States, Canada)
kmod-nls-cp775 Kernel module for NLS Codepage 775 (Baltic Rim)
kmod-nls-cp850 Kernel module for NLS Codepage 850 (Europe)
kmod-nls-cp852 Kernel module for NLS Codepage 852 (Europe)
kmod-nls-cp866 Kernel module for NLS Codepage 866 (Cyrillic)
kmod-nls-iso8859-1 Kernel module for NLS ISO 8859-1 (Latin 1)
kmod-nls-iso8859-13 Kernel module for NLS ISO 8859-13 (Latin 7; Baltic)
kmod-nls-iso8859-15 Kernel module for NLS ISO 8859-15 (Latin 9)
kmod-nls-iso8859-2 Kernel module for NLS ISO 8859-2 (Latin 2)
kmod-nls-koi8r Kernel module for NLS KOI8-R (Russian)
kmod-nls-utf8 Kernel module for NLS UTF-8

Formatting and File System Checking from within OpenWrt

The package e2fsprogs contains the essential ext2/3/4 filesystem utilities e2fsck, mke2fs, debugfs, dumpe2fs and tune2fs, and also most of the other core ext2/3/4 filesystem utilities available. For help with formatting see TLDP. Also see this page and the official page for more information on e2fsprogs.

Example

As an example, for a device situated at /dev/sda, to format its second partition /dev/sda2 with the ext4 file system you can run

  mkfs.ext4 /dev/sda2

:!: Note: When doing this, there will be no warning before execution and a new empty ext4 file system will be written to /dev/sda2 immediately, using a quick format. Any data on the partition cannot be easily retrieved or is lost. Make sure that you enter the correct device file and partition as an argument for the formatting tool.

File System Checking

The file system checking utilities can be used in case the file system requires checking for errors.

:!: Note: In order to perform a e2fsck on 1 GiB partition, you will need at least 1 MiB of RAM.2) The same is true for mke2fs. You can solve this by mounting a SWAP partition (or by using a device with enough RAM).

:!: Note: In order to do fsck (aka CHKDSK) of FAT32/VFAT partitions before mounting (i.e. during the boot process), you should read the following tickets: Incorrect log output for mount point file systems without a fsck, and Missing fsck function for block-mount/-hotplug of dos/vfat filesystems

Mounting

There is no additional configuration needed for mounting your file system when the device is properly configured. Mounting works as is typical in Linux. Read the manual page for mount. Pay special attention to the options.

You can only mount a partition to an existing directory. You can create it with mkdir, for example:

mkdir -p /mnt/shares

To manually mount a partition, assuming that the second partition /dev/sda2 on the device /dev/sda is formatted with an ext4 file system, use:

mount -t ext4 /dev/sda2 /mnt/shares 

The device can now be accessed at /mnt/shares. Note that the -t ext4 option is not strictly necessary as the file system is usually automatically detected.

To check if mounting was successful use df:

root@OpenWrt:~# df -h
Filesystem                Size      Used Available Use% Mounted on
rootfs                   28.8M      4.5M     24.3M  16% /
/dev/root                 2.0M      2.0M         0 100% /rom
tmpfs                    61.8M    400.0K     61.4M   1% /tmp
/dev/mtdblock5           28.8M      4.5M     24.3M  16% /overlay
overlayfs:/overlay       28.8M      4.5M     24.3M  16% /
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/sda2                18.7G     67.6M     13.9G   0% /mnt/shares
The device is listed here with its mount point in the last line. You can also run mount without arguments to list all mount points.

To unmount:

umount /mnt/shares

To mount a swap partition, assuming that the first partition on the /dev/sda device is linux swap space:

swapon /dev/sda1

In case /dev/sda1 is not swap space yet, you can run

mkswap /dev/sda1

You can check swap functionality and status with the free command. You can also check its status using top or htop (installable package).

To unmount a swap partition:

swapoff /dev/sda1

Usually the devices that you use are block devices. In OpenWrt, extra utilities for block devices such as automatic mounting are provided with a collection of tools for block devices, see also below.

Mount on Boot

It is convenient to be able to mount your file systems on boot. The package block-mount will take care of this and also enable automatic mount when device is plugged in through use of ubox. As is typical in Linux, this is handled with Fstab Configuration. OpenWrt provides UCI configuration of Fstab. The configuration file is /etc/config/fstab.

For a technical background in which Fstab works, see block_mount. Note the information box below mentioning the new block-mount packages which works in conjunction with ubox.

Example settings in fstab for mounting to /mnt/share with automatic detection of the file system:

config 'mount' option 'device' '/dev/sda1' option 'options' 'rw,sync' option 'enabled_fsck' '0' option 'enabled' '1' option 'target' '/mnt/share'

And to make the fstab settings effective in current session run:

/etc/init.d/fstab start

To make the fstab settings effective after every reboot run:

/etc/init.d/fstab enable

:!: Note: If you are having trouble with fstab, see fstab troubleshooting.

UCI CLI commands:

/etc/init.d/fstab stop
uci add fstab mount
uci set fstab.@mount[-1].device=/dev/sda1
uci set fstab.@mount[-1].options=rw,sync
uci set fstab.@mount[-1].enabled_fsck=0
uci set fstab.@mount[-1].enabled=1
uci set fstab.@mount[-1].target=/mnt/share
uci commit fstab
/etc/init.d/fstab start

Delete with UCI CLI commands:

/etc/init.d/fstab stop
uci delete fstab.@mount[0]
uci commit fstab
/etc/init.d/fstab start

In r26314 the three opkg packages block-mount, block-extroot and block-hotplug have been merged into block-mount.
In r36988 ubox is a dependency of the opkg package block-mount. block-mount now contains the executable block. block <info|mount|umount|detect>
2013-07-04: new block mount howto

Installing OPKG packages in mount point other than root

Once you successfully mounted your partition, you may want to install packages onto it. Please see Installation.Destination for that.

If you plan to install kernel modules on a USB drive you might want to read this: https://dev.openwrt.org/ticket/10739

An even better option might be to use extroot. This will allow you to use your device as a root, by using it in an overlay configuration over the original root.

Automatic spin down an attached hard disk

If you have attached an external hard disk (e.g. through the USB port) and you don't use the disk all the time, you may want to spin down the disk after some time. One possible reason for this is to reduce the noise. A second reason is that some fanless external racks can get very hot and this is not healthy for the disk.

Unfortunately there is no solution that works always for everybody. The first option to try is provided by the package ''hd-idle'', which will spin down any disk accessible via the SCSI layer. Install it and test if it can spindown the hard disk (let us assume that your disk is /dev/sda):

opkg install hd-idle
hd-idle -t sda
If you can hear the disk spins down, then you can configure the package, the configuration is in /etc/config/hd-idle.

If hd-idle doesn't work, then the next option to try is hdparm:

opkg install hdparm
hdparm -y /dev/sda

If you hear the disk spins down, then create a script /etc/spindown with the following contents:

#!/bin/sh

oldstate=''
n="$2"
while :; do
	sleep 60
	newstate=$(grep "$1" /proc/diskstats) 
	if [ "$newstate" = "$oldstate" ]; then
		n=$(($n - 1))
		[ "$n" != 0 ] || hdparm -y /dev/"$1"
	else
		n="$2"
		oldstate=$newstate
	fi
done &
Then add to /etc/rc.local the following command (assuming your disk is /dev/sda and you want to spin it down after 20 minutes of inactivity):
/etc/spindown sda 20
A much easier way is this: if the above command hdparm -y /dev/sda works, insert the following command to /etc/rc.local:
hdparm -S 240 /dev/hda
The setting 240 equals 20 mins, the time coding can be checked on the ''hdparm man page''.

The third option to try is the package sdparm:

opkg install sdparm
sdparm -f -C stop /dev/sda
If you hear the disk spins down, use the script /etc/spindown as in the case of hdparm, however replace the line
[ "$n" != 0 ] || hdparm -y /dev/"$1"
with the line
[ "$n" != 0 ] || sdparm -f -C stop /dev/"$1"

Virtual Disk

Now let's do a virtual disk. You need an additional package.

 
opkg update
opkg install kmod-fs-ext3 kmod-loop e2fsprogs
insmod ext3
insmod loop

Then, create, and format the disk:

dd if=/dev/zero of=/mnt/hdd/owrt_loop bs=1M count=128
mke2fs -j -F -b 1024 /mnt/hdd/owrt_loop

To be able to use, plug-in for OpenWrt:

mkdir /mnt/loopdisk
mount -o loop /mnt/hdd/owrt_loop /mnt/loopdisk

Now our system is not only the OpenWrt xMB, but x+128 MB

Write in /etc/opkg.conf file into the following line

#installation.destinations

dest usb /mnt/loopdisk

To install package:

opkg -dest usb install package

To delete package:

opkg -dest usb remove package

Troubleshooting

I have problems with FAT or NTFS file systems

There are numerous things that go wrong with file systems not made for Linux. ext2, ext3 or ext4 file systems are therefore preferred.

kmod-fs-ext4 won't mount ext2 or ext3 partition

Symptoms: You are trying to mount ext2 or ext3 partition and you have installed only kmod-fs-ext4. Neither mount command nor fstab works. The kmod-fs-ext4 issue definitely applies to Backfire, but is not an issue for Attitude Adjustment (tested on Attitude Adjustment RC1).

Solution a): you have to make sure that you are trying to mount the ext2 or ext3 partition like it was ext4 partition. Example /etc/config/fstab config and mount command follows:

config mount option target /mnt option device /dev/sda1 # sda1 is ext3 partition option fstype ext4 # but is mounted as ext4, since we have installed kmod-fs-ext4 only option options rw,sync option enabled 1 option enabled_fsck 1

mount -t ext4 /dev/sda1 /mnt -o rw,sync

Solution b): install kmod-fs-ext3 or kmod-fs-ext2 filesystem.

opkg update
opkg install kmod-fs-ext3

:!: This only applies to Backfire

Other Useful Packages

  • swap-utils This package contains a collection of tools for managing swap space: mkswap, swapon and swapoff. FIXME This package exists, but swapon and swapoff seem to be just built into Busybox. Can someone explain this here?
  • blkid Command-line utility to locate/print block device attributes.
  • dd A bit-stream duplicator to write directly to block devices.

Notes

:!: Note: If you are having trouble using storage on a Huawei 3G dongle, you should read the following ticket:Mass storage missing from 3G dongle

1) But then on Backfire ext2 or ext3 partition has to be mounted as ext4 partition, see Troubleshooting.

Back to top

doc/howto/storage.txt · Last modified: 2014/02/15 23:08 by seprodib