Differences

This shows you the differences between two versions of the page.

doc:techref:filesystems [2012/10/29 10:23]
brian rewritten for clarity
doc:techref:filesystems [2014/02/04 07:04] (current)
hughredelmeier mention that it is useful for NAND flash
Line 1: Line 1:
====== Filesystems ====== ====== Filesystems ======
-Please read about the -> [[flash.layout]] as well. Also, note that there are two types of flash memory: [[wp>Flash_memory#NOR_flash|NOR flash]] and [[wp>Flash_memory#NAND_flash|NAND flash]]. Also, you should definitely read up on ''[[doc:techref:mtd]]''.+Please read about the -> [[flash.layout]] as well. Also, note that there are two types of flash memory: [[wp>Flash_memory#NOR_flash|NOR flash]] and [[wp>Flash_memory#NAND_flash|NAND flash]]. Also, you should read up on ''[[doc:techref:mtd]]''. This article is about file systems in the OpenWrt installation on built-in flash. For general external support for installing file systems on other devices, including partitioning and mounting see [[doc/howto/storage|this page about general storage]].
-===== mini_fo =====+===== mini_fo (replaced by overlayfs) =====
The (mini fanout overlay file system) – Redirects modifying operations to a writable location called "storage directory", and leaving the original data in the "base directory" untouched. When reading, the file system merges the modified and original data so that only the newest versions will appear. The (mini fanout overlay file system) – Redirects modifying operations to a writable location called "storage directory", and leaving the original data in the "base directory" untouched. When reading, the file system merges the modified and original data so that only the newest versions will appear.
  * [[http://lwn.net/Articles/135283/]]   * [[http://lwn.net/Articles/135283/]]
Line 9: Line 9:
===== overlayfs ===== ===== overlayfs =====
 +Used to merge two filesystems, one read-only and the other writable.  [[flash.layout]] explains how this is used in OpenWRT.
  * [[https://dev.openwrt.org/browser/trunk/target/linux/generic/patches-2.6.38/209-overlayfs.patch?rev=26213]]   * [[https://dev.openwrt.org/browser/trunk/target/linux/generic/patches-2.6.38/209-overlayfs.patch?rev=26213]]
  * http://lwn.net/Articles/447650/   * http://lwn.net/Articles/447650/
 +  * mainlined in Linux kernel 3.11, see [[https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/Documentation/filesystems/overlayfs.txt|/Documentation/filesystems/overlayfs.txt]] <- not mainlined yet, empty file?
 +  * [[https://git.kernel.org/cgit/linux/kernel/git/mszeredi/vfs.git/tree/Documentation/filesystems/overlayfs.txt?h=overlayfs.current | Overlayfs documentation]] in the official development tree
 +  * Overlayfs's support for inotify mechanisms is not complete yet. Events like IN_CLOSE_WRITE cannot be notified to listening process.
===== SquashFS ===== ===== SquashFS =====
Line 22: Line 26:
  * [[http://lxr.free-electrons.com/source/Documentation/filesystems/squashfs.txt|Kernel documentation on SquashFS]]   * [[http://lxr.free-electrons.com/source/Documentation/filesystems/squashfs.txt|Kernel documentation on SquashFS]]
  * [[http://tree.celinuxforum.org/CelfPubWiki/SquashFsComparisons|SquashFs Performance Comparisons]]   * [[http://tree.celinuxforum.org/CelfPubWiki/SquashFsComparisons|SquashFs Performance Comparisons]]
 +  * There is a generic problem when running SquashFS on NAND: The issue is that SquashFS has no bad block management at all and requires all blocks on order; but for proper NAND bad block management you also need to be able to skip bad blocks and occasionally relocate blocks (see [[http://www.infradead.org/pipermail/linux-mtd/2006-April/015386.html|squashfs and NAND flash]]). That's why raw SquashFS is a bad idea on NAND (it works if you use a FTL like UBIFS).
 +
===== JFFS2 ===== ===== JFFS2 =====
Line 28: Line 34:
  * + is writable, has journaling and wear leveling   * + is writable, has journaling and wear leveling
  * + is cool   * + is cool
-  * - is compressed, so a program (''[[doc:techref:opkg]]'' in particularly) cannot know in advance how much space a package will occupy+  * - is compressed, so a program (''[[doc:techref:opkg]]'' in particular) cannot know in advance how much space a package will occupy
===== UBIFS ===== ===== UBIFS =====
-[[wp>UBIFS]]+[[wp>UBIFS]] is a file system for [[doc/techref/flash.layout|raw flash]].  It is a good choice for NAND flash (but OpenWRT isn't set up for this).
  * [[http://lxr.free-electrons.com/source/Documentation/filesystems/ubifs.txt|Kernel documentation on UBIFS]]   * [[http://lxr.free-electrons.com/source/Documentation/filesystems/ubifs.txt|Kernel documentation on UBIFS]]
Line 39: Line 45:
===== CramFS ===== ===== CramFS =====
-[[wp>CramFS]] is not utilized by OpenWrt, but the [[toh/inventel/dv4210|DV4210 (Livebox)]] uses it.+[[wp>CramFS]] is not utilized by OpenWrt.
  * [[http://lxr.free-electrons.com/source/Documentation/filesystems/cramfs.txt|Kernel documentation on CramFS]]   * [[http://lxr.free-electrons.com/source/Documentation/filesystems/cramfs.txt|Kernel documentation on CramFS]]
Line 62: Line 68:
===== Implementation in OpenWrt ===== ===== Implementation in OpenWrt =====
-The [[Flash Layout]] article documents how OpenWrt uses both SquashFS and JFFS2 filesystems at the same time. The kernel is also stored separately from these partitions in raw flash. When the kernel is built, it is also compressed with [[wp>Lempel–Ziv–Markov chain algorithm|LZMA]] and [[wp>gzip]], as documented in [[doc:howto:obtain.firmware.generate]].+The [[Flash Layout]] article documents how OpenWrt uses both SquashFS and JFFS2 filesystems combined into one filesystem by overlayfs. The kernel is also stored separately from these partitions in raw flash. When the kernel is built, it is also compressed with [[wp>Lempel–Ziv–Markov chain algorithm|LZMA]] and [[wp>gzip]], as documented in [[doc:howto:obtain.firmware.generate]].
==== Boot process ==== ==== Boot process ====
System bootup is as follows: ->[[process.boot]] System bootup is as follows: ->[[process.boot]]
-  - kernel boots from SquashFS and runs ''[[doc:howto:notuci.config#etcpreinit|/etc/preinit]]''+  - kernel boots from a known raw partition (without a FS), scans mtd partition //rootfs// for a valid superblock and mounts the SquashFS partition (containing ''/etc'') then runs ''[[doc:howto:notuci.config#etcpreinit|/etc/preinit]]''. (More info at [[doc/techref/filesystems#technical.details]])
  - ''/etc/preinit'' runs ''[[https://dev.openwrt.org/browser/trunk/package/base-files/files/sbin/mount_root|/sbin/mount_root]]''   - ''/etc/preinit'' runs ''[[https://dev.openwrt.org/browser/trunk/package/base-files/files/sbin/mount_root|/sbin/mount_root]]''
-  - ''mount_root'' mounts the JFFS2 partition (''/jffs'') and **combines** it with the SquashFS partition (''/rom'') to create a new //virtual root filesystem// (''/'')+  - ''mount_root'' mounts the JFFS2 partition (''/overlay'') and **combines** it with the SquashFS partition (''/rom'') to create a new //virtual root filesystem// (''/'')
  - bootup continues with ''/sbin/init''   - bootup continues with ''/sbin/init''
 +** * ** ''/overlay'' was previously named ''/jffs2''
==== Explanation ==== ==== Explanation ====
Line 76: Line 82:
Both SquashFS and JFFS2 are compressed filesystems using [[wp>Lempel–Ziv–Markov chain algorithm|LZMA]] for the compression. SquashFS is a //read only// filesystem while JFFS2 is a writable filesystem with //journaling// and //wear leveling//. Both SquashFS and JFFS2 are compressed filesystems using [[wp>Lempel–Ziv–Markov chain algorithm|LZMA]] for the compression. SquashFS is a //read only// filesystem while JFFS2 is a writable filesystem with //journaling// and //wear leveling//.
-Our job when writing the firmware is to put as much common functionality on SquashFS while not wasting space with unwanted features. Additional features can always be installed onto JFFS2 by the user. The use of ''mini_fo'' means that the filesystem is presented as one large writable filesystem to the user with no visible boundary between SquashFS and JFFS2 -- files are simply copied to JFFS2 when they're written.+Our job when writing the firmware is to put as much common functionality on SquashFS while not wasting space with unwanted features. Additional features can always be installed onto JFFS2 by the user. The use of ''mini_fo''/''overlayfs'' means that the filesystem is presented as one large writable filesystem to the user with no visible boundary between SquashFS and JFFS2 -- files are simply copied to JFFS2 when they're written.
It's not all without side effects however - It's not all without side effects however -
Line 87: Line 93:
==== Explanation 2 ==== ==== Explanation 2 ====
-On many embedded targets that use  [[wp>Flash_memory#NAND_flash|NAND flash]] for the root filesystem, OpenWrt implements a clever trick to get the most out of the limited flash memory capacity while retaining flexibility for the end-user:+On many embedded targets that use  [[wp>Flash_memory#NOR_flash|NOR flash]] for the root filesystem, OpenWrt implements a clever trick to get the most out of the limited flash memory capacity while retaining flexibility for the end-user:
Basically, during the image creation, all of the rootfs contents is packed up in a SquashFS filesystem -- a highly efficient filesystem Basically, during the image creation, all of the rootfs contents is packed up in a SquashFS filesystem -- a highly efficient filesystem
-with compression support. There's one important detail about it though: it is a read-only filesystem. To overcome this limitation OpenWrt uses the remaining portion of the NAND rootfs partition to store an additional read/write jffs2 filesystem which is "overlayed" on top of the rootfs (that is, allowing to read unchanged files from the SquashFS but storing all the modifications made to the jffs2 part).+with compression support. There's one important detail about it though: it is a read-only filesystem. To overcome this limitation OpenWrt uses the remaining portion of the NOR rootfs partition to store an additional read/write jffs2 filesystem which is "overlayed" on top of the rootfs (that is, allowing to read unchanged files from the SquashFS but storing all the modifications made to the jffs2 part).
This design has another important advantage for the end-user: even This design has another important advantage for the end-user: even
Line 100: Line 106:
==== Technical Details ==== ==== Technical Details ====
-The kernel boot process involves discovering of NAND partitions (what is a "NAND partition"?) and it can be done by various target-dependent means:+The kernel boot process involves discovering of partitions within the NOR flash and it can be done by various target-dependent means:
  * some bootloaders store a partition table at a known location   * some bootloaders store a partition table at a known location
  * some pass the partition layout via kernel command line   * some pass the partition layout via kernel command line

Back to top

doc/techref/filesystems.1351502638.txt.bz2 · Last modified: 2012/10/29 10:23 by brian