Differences

This shows you the differences between two versions of the page.

doc:techref:filesystems [2012/10/29 10:23]
brian rewritten for clarity
doc:techref:filesystems [2014/07/06 13:59] (current)
theoradicus mention ubifs only on NAND
Line 1: Line 1:
====== Filesystems ====== ====== Filesystems ======
-Please read about the -> [[flash.layout]] as well. Also, note that there are two types of flash memory: [[wp>Flash_memory#NOR_flash|NOR flash]] and [[wp>Flash_memory#NAND_flash|NAND flash]]. Also, you should definitely read up on ''[[doc:techref:mtd]]''. 
-===== mini_fo ===== +This article is about file systems in the OpenWrt installation on built-in flash. 
-The (mini fanout overlay file system) – Redirects modifying operations to a writable location called "storage directory", and leaving the original data in the "base directory" untouched. When reading, the file system merges the modified and original data so that only the newest versions will appear. +For general external support for installing file systems on other devices, including partitioning and mounting see [[doc/howto/storage|this page about general storage]]
-  * [[http://lwn.net/Articles/135283/]] + 
- * [[http://www.denx.de/wiki/Know/MiniFOHome|Homepage]] +Please read about the -> [[flash.layout]] as well. Also, note that there are two types of flash memory: [[wp>Flash_memory#NOR_flash|NOR flash]] and [[wp>Flash_memory#NAND_flash|NAND flash]]. Also, you should read up on ''[[doc:techref:mtd]]''.  
-  * was replaced with overlayfs, see [[https://dev.openwrt.org/search?q=overlayfs&noquickjump=1&changeset=on&milestone=on&wiki=on|dev]]+ 
===== overlayfs ===== ===== overlayfs =====
 +Used to merge two filesystems, one read-only and the other writable.  [[flash.layout]] explains how this is used in OpenWRT.
  * [[https://dev.openwrt.org/browser/trunk/target/linux/generic/patches-2.6.38/209-overlayfs.patch?rev=26213]]   * [[https://dev.openwrt.org/browser/trunk/target/linux/generic/patches-2.6.38/209-overlayfs.patch?rev=26213]]
  * http://lwn.net/Articles/447650/   * http://lwn.net/Articles/447650/
 +  * mainlined in Linux kernel 3.11, see [[https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/Documentation/filesystems/overlayfs.txt|/Documentation/filesystems/overlayfs.txt]] <- not mainlined yet, empty file?
 +  * [[https://git.kernel.org/cgit/linux/kernel/git/mszeredi/vfs.git/tree/Documentation/filesystems/overlayfs.txt?h=overlayfs.current | Overlayfs documentation]] in the official development tree
 +  * Overlayfs's support for inotify mechanisms is not complete yet. Events like IN_CLOSE_WRITE cannot be notified to listening process.
 +
 +===== tmpfs =====
 +[[wp>tmpfs]]
 +  * ''/tmp'' resides on a tmpfs-partition and ''/var'' is a symlink to it; ''/dev'' resides on a little tmpfs partition of its own
 +  * + no wear leveling
 +  * - volatile (doesn't survive a reboot)
 +  * [[http://lxr.free-electrons.com/source/Documentation/filesystems/tmpfs.txt|Kernel documentation on tmpfs]]
===== SquashFS ===== ===== SquashFS =====
Line 22: Line 33:
  * [[http://lxr.free-electrons.com/source/Documentation/filesystems/squashfs.txt|Kernel documentation on SquashFS]]   * [[http://lxr.free-electrons.com/source/Documentation/filesystems/squashfs.txt|Kernel documentation on SquashFS]]
  * [[http://tree.celinuxforum.org/CelfPubWiki/SquashFsComparisons|SquashFs Performance Comparisons]]   * [[http://tree.celinuxforum.org/CelfPubWiki/SquashFsComparisons|SquashFs Performance Comparisons]]
 +  * There is a generic problem when running SquashFS on NAND: The issue is that SquashFS has no bad block management at all and requires all blocks on order; but for proper NAND bad block management you also need to be able to skip bad blocks and occasionally relocate blocks (see [[http://www.infradead.org/pipermail/linux-mtd/2006-April/015386.html|squashfs and NAND flash]]). That's why raw SquashFS is a bad idea on NAND (it works if you use a FTL like UBIFS).
 +
===== JFFS2 ===== ===== JFFS2 =====
Line 28: Line 41:
  * + is writable, has journaling and wear leveling   * + is writable, has journaling and wear leveling
  * + is cool   * + is cool
-  * - is compressed, so a program (''[[doc:techref:opkg]]'' in particularly) cannot know in advance how much space a package will occupy+  * - is compressed, so a program (''[[doc:techref:opkg]]'' in particular) cannot know in advance how much space a package will occupy 
 +  * + is compressed, so a program (''[[doc:techref:opkg]]'' which is preinstalled takes much less space, so effectively you have more space
===== UBIFS ===== ===== UBIFS =====
-[[wp>UBIFS]]+[[wp>UBIFS]] is a file system for [[doc/techref/flash.layout|raw flash]]. It is used in OpenWrt NAND targets since :FIXME: around r40364
  * [[http://lxr.free-electrons.com/source/Documentation/filesystems/ubifs.txt|Kernel documentation on UBIFS]]   * [[http://lxr.free-electrons.com/source/Documentation/filesystems/ubifs.txt|Kernel documentation on UBIFS]]
- 
-===== YAFFS2 ===== 
-[[wp>YAFFS]] 
- 
-===== CramFS ===== 
-[[wp>CramFS]] is not utilized by OpenWrt, but the [[toh/inventel/dv4210|DV4210 (Livebox)]] uses it. 
- 
-  * [[http://lxr.free-electrons.com/source/Documentation/filesystems/cramfs.txt|Kernel documentation on CramFS]] 
===== ext2 ====== ===== ext2 ======
 +
 +Ext2/3/4 is used on x86, x86-64 and for some arch with SD-card rootfs :FIXME: .
[[wp>ext2]] [[wp>ext2]]
  * + a program (''[[doc:techref:opkg]]'' in particularly) knows how much space is left!   * + a program (''[[doc:techref:opkg]]'' in particularly) knows how much space is left!
Line 53: Line 61:
  * [[http://lxr.free-electrons.com/source/Documentation/filesystems/ext2.txt|Kernel documentation on ext2]]   * [[http://lxr.free-electrons.com/source/Documentation/filesystems/ext2.txt|Kernel documentation on ext2]]
-===== tmpfs ===== 
-[[wp>tmpfs]] 
-  * ''/tmp'' resides on a tmpfs-partition and ''/var'' is a symlink to it; ''/dev'' resides on a little tmpfs partition of its own 
-  * + no wear leveling 
-  * - volatile (doesn't survive a reboot) 
-  * [[http://lxr.free-electrons.com/source/Documentation/filesystems/tmpfs.txt|Kernel documentation on tmpfs]] 
 +===== Other filesystems =====
 +
 +OpenWrt does not use other filesystems as rootfs. It supports several filesystem attached to via various mechanisms like USB,SATA or network. For a list see [[doc:howto:storage]].
===== Implementation in OpenWrt ===== ===== Implementation in OpenWrt =====
-The [[Flash Layout]] article documents how OpenWrt uses both SquashFS and JFFS2 filesystems at the same time. The kernel is also stored separately from these partitions in raw flash. When the kernel is built, it is also compressed with [[wp>Lempel–Ziv–Markov chain algorithm|LZMA]] and [[wp>gzip]], as documented in [[doc:howto:obtain.firmware.generate]].+The [[Flash Layout]] article documents how OpenWrt uses both SquashFS and JFFS2 filesystems combined into one filesystem by overlayfs. The kernel is also stored separately from these partitions in raw flash. When the kernel is built, it is also compressed with [[wp>Lempel–Ziv–Markov chain algorithm|LZMA]] and [[wp>gzip]], as documented in [[doc:howto:obtain.firmware.generate]].
==== Boot process ==== ==== Boot process ====
System bootup is as follows: ->[[process.boot]] System bootup is as follows: ->[[process.boot]]
-  - kernel boots from SquashFS and runs ''[[doc:howto:notuci.config#etcpreinit|/etc/preinit]]''+  - kernel boots from a known raw partition (without a FS), scans mtd partition //rootfs// for a valid superblock and mounts the SquashFS partition (containing ''/etc'') then runs ''[[doc:howto:notuci.config#etcpreinit|/etc/preinit]]''. (More info at [[doc/techref/filesystems#technical.details]])
  - ''/etc/preinit'' runs ''[[https://dev.openwrt.org/browser/trunk/package/base-files/files/sbin/mount_root|/sbin/mount_root]]''   - ''/etc/preinit'' runs ''[[https://dev.openwrt.org/browser/trunk/package/base-files/files/sbin/mount_root|/sbin/mount_root]]''
-  - ''mount_root'' mounts the JFFS2 partition (''/jffs'') and **combines** it with the SquashFS partition (''/rom'') to create a new //virtual root filesystem// (''/'')+  - ''mount_root'' mounts the JFFS2 partition (''/overlay'') and **combines** it with the SquashFS partition (''/rom'') to create a new //virtual root filesystem// (''/'')
  - bootup continues with ''/sbin/init''   - bootup continues with ''/sbin/init''
 +** * ** ''/overlay'' was previously named ''/jffs2''
==== Explanation ==== ==== Explanation ====
Line 76: Line 81:
Both SquashFS and JFFS2 are compressed filesystems using [[wp>Lempel–Ziv–Markov chain algorithm|LZMA]] for the compression. SquashFS is a //read only// filesystem while JFFS2 is a writable filesystem with //journaling// and //wear leveling//. Both SquashFS and JFFS2 are compressed filesystems using [[wp>Lempel–Ziv–Markov chain algorithm|LZMA]] for the compression. SquashFS is a //read only// filesystem while JFFS2 is a writable filesystem with //journaling// and //wear leveling//.
-Our job when writing the firmware is to put as much common functionality on SquashFS while not wasting space with unwanted features. Additional features can always be installed onto JFFS2 by the user. The use of ''mini_fo'' means that the filesystem is presented as one large writable filesystem to the user with no visible boundary between SquashFS and JFFS2 -- files are simply copied to JFFS2 when they're written.+Our job when writing the firmware is to put as much common functionality on SquashFS while not wasting space with unwanted features. Additional features can always be installed onto JFFS2 by the user. The use of ''mini_fo''/''overlayfs'' means that the filesystem is presented as one large writable filesystem to the user with no visible boundary between SquashFS and JFFS2 -- files are simply copied to JFFS2 when they're written.
It's not all without side effects however - It's not all without side effects however -
Line 87: Line 92:
==== Explanation 2 ==== ==== Explanation 2 ====
-On many embedded targets that use  [[wp>Flash_memory#NAND_flash|NAND flash]] for the root filesystem, OpenWrt implements a clever trick to get the most out of the limited flash memory capacity while retaining flexibility for the end-user:+On many embedded targets that use  [[wp>Flash_memory#NOR_flash|NOR flash]] for the root filesystem, OpenWrt implements a clever trick to get the most out of the limited flash memory capacity while retaining flexibility for the end-user:
Basically, during the image creation, all of the rootfs contents is packed up in a SquashFS filesystem -- a highly efficient filesystem Basically, during the image creation, all of the rootfs contents is packed up in a SquashFS filesystem -- a highly efficient filesystem
-with compression support. There's one important detail about it though: it is a read-only filesystem. To overcome this limitation OpenWrt uses the remaining portion of the NAND rootfs partition to store an additional read/write jffs2 filesystem which is "overlayed" on top of the rootfs (that is, allowing to read unchanged files from the SquashFS but storing all the modifications made to the jffs2 part).+with compression support. There's one important detail about it though: it is a read-only filesystem. To overcome this limitation OpenWrt uses the remaining portion of the NOR rootfs partition to store an additional read/write jffs2 filesystem which is "overlayed" on top of the rootfs (that is, allowing to read unchanged files from the SquashFS but storing all the modifications made to the jffs2 part).
This design has another important advantage for the end-user: even This design has another important advantage for the end-user: even
Line 100: Line 105:
==== Technical Details ==== ==== Technical Details ====
-The kernel boot process involves discovering of NAND partitions (what is a "NAND partition"?) and it can be done by various target-dependent means:+The kernel boot process involves discovering of partitions within the NOR flash and it can be done by various target-dependent means:
  * some bootloaders store a partition table at a known location   * some bootloaders store a partition table at a known location
  * some pass the partition layout via kernel command line   * some pass the partition layout via kernel command line
Line 136: Line 141:
  * how data is stored and addressed by SquashFS:   * how data is stored and addressed by SquashFS:
  * how data is stored and addressed by JFFS2:   * how data is stored and addressed by JFFS2:
 +
 +===== Archive =====
 +
 +  * see [[doc:techref:filesystems.old]]

Back to top

doc/techref/filesystems.1351502638.txt.bz2 · Last modified: 2012/10/29 10:23 by brian