User Tools

Site Tools


doc:techref:flash.layout

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
doc:techref:flash.layout [2013/02/06 17:20]
buntalo
doc:techref:flash.layout [2014/08/09 01:09] (current)
hawkse Linked the Flash layout document to an archived copy at archive.org since the original link disappeared.
Line 1: Line 1:
 ====== The OpenWrt Flash Layout ====== ====== The OpenWrt Flash Layout ======
-There are hard discs, which are considered block devices, ​and there is flash memory. There types of flash, like NOR flash, SLC NAND flash and MLC NAND flash.\\ + 
-If the flash chip (no matter what type) is connected directly with the [[http://​en.wikipedia.org/​wiki/​System-on-a-chip|SoC]] and has to be addressed directly by Linux, we call it **//"​raw flash"//​**. +Most routers do not have hard drives. ​ They use flash memory for similar purposes: storing programs ​and data, even when the system ​is off (non-volatile memory). 
-If the flash chip cannot be addressed directly by the OS (because ​there is an additional controller chip between flash chip and the SoC), we call it **//"​FTL (Flash Translation Layer) flash"//​**. Embedded systems almost exclusively use "raw flash",​ while USB memory sticks, almost exclusively use FTL flash!+ 
 +In most systems, ​flash memory ​does not appear like RAM and so data and instructions must be copied to RAM to be used So, for example, the bootloader copies the kernel from flash to RAM and then starts that copy running. 
 + 
 +There are two basic types of flash memory: [[wp>​Flash_memory#​NOR_flash|NOR flash]] and [[wp>​Flash_memory#​NAND_flash|NAND flash]].\\ 
 +If the flash chip (no matter what type) is connected directly with the [[wp>System-on-a-chip|SoC]] and has to be addressed directly by Linux, we call it **//"​raw flash"//​**. 
 +If there is an additional controller chip between flash chip and the SoC, we call it **//"​FTL (Flash Translation Layer) flash"//​**. Embedded systems almost exclusively use "raw flash",​ while SSDs and also USB memory sticks, almost exclusively use FTL flash! 
 + 
 +Older routers generally have raw NOR flash but many newer routers have raw NAND flash. 
 + 
 +Raw NOR flash in typical routers is generally small (4 MiB - 16 MiB) and error-free. ​ This is currently the normal OpenWRT setup. ​ Because it is error-free, the file systems are a little simpler. ​ But they still need to do wear-levelling. ​ Conserving space is important. 
 + 
 +Raw NAND flash in typical routers is generally larger (32 MiB - 256 MiB) and may contain bad blocks. ​ OpenWRT isn't configured to handle this yet.  One promising approach would be to use [[https://​en.wikipedia.org/​wiki/​UBIFS|UBIFS]] layered on top of UBI.
  
 ===== Partitioning of SquashFS-Images ===== ===== Partitioning of SquashFS-Images =====
-The flash chip on embedded systems ​is not accompanied by a controller chip, and thus are not //"​FTL"//​-devices but //"raw flash"//​-devicesStorage ​space is treated and addressed ​as an [[doc:​techref:​MTD| MTD (Memory Technology Device)]] and special ​[[doc:​techref:​Filesystems]] are used. The available storage is not partitioned in the traditional way, where you store the data about the partitions in the [[wp>​Master boot record|MBR]] and [[wp>​Volume boot record|PBR]]s,​ but it is done in the Linux Kernel (and sometimes independently in the [[doc:​techref:​bootloader]] as well!). You simply define, that "​partition **//​kernel//​** starts at offset x and ends at offset y". The advantage is, that later we can address these partitions by name, rather then specifying the precise start point of the data.+Almost all embedded systems ​contain ​//"raw flash"//​-chipsOpenWrt treats and addresses this storage ​space as an [[doc:​techref:​MTD| MTD (Memory Technology Device)]] and employs ​[[doc:​techref:​filesystems]] developed for this purpose. The available storage is not partitioned in the traditional way, where you store the data about the partitions in the [[wp>​Master boot record|MBR]] and [[wp>​Volume boot record|PBR]]s,​ but it is done in the Linux Kernel (and sometimes independently in the [[doc:​techref:​bootloader]] as well!). You simply define, that "​partition **//​kernel//​** starts at offset x and ends at offset y". The advantage is, that later we can address these partitions by name, rather then specifying the precise start point of the data.
  
-:!: The following table documents ​the layout of the [[toh:​tp-link:​TL-WR1043ND]] and is merely one **example**! Another, slightly different, example can be found on the wiki-page for the [[toh:​d-link:​DIR-300#​flash.layout|DIR-300]]. So the layout does differ between the devices! Please see the wiki-page for a device, for information about its particular layout and simply check it yourself on your device.+:!: The following table gives you a visual of the layout of the [[toh:​tp-link:​TL-WR1043ND]] and is merely one **example**! Another, slightly different, example can be found on the wiki-page for the [[toh:​d-link:​DIR-300#​flash.layout|DIR-300]]. So the layout does differ between the devices! Please see the wiki-page for a device, for information about its particular layout and simply check it yourself on your device.
  
 ^   ​TP-Link WR1043ND ​ Flash Layout ​          ​^^^^^^ ^   ​TP-Link WR1043ND ​ Flash Layout ​          ​^^^^^^
Line 20: Line 31:
 ^ <color magenta>​mountpoint</​color> ​  ​| ​ //​none// ​                   |  //​none// ​                    ​| ​ <color magenta>''/​rom''</​color> ​ |  <color magenta>''/​overlay''</​color> ​  ​| ​ //​none// ​ | ^ <color magenta>​mountpoint</​color> ​  ​| ​ //​none// ​                   |  //​none// ​                    ​| ​ <color magenta>''/​rom''</​color> ​ |  <color magenta>''/​overlay''</​color> ​  ​| ​ //​none// ​ |
 ^ filesystem ​  ​| ​ //​none// ​                   |  //​none// ​                    ​| ​ [[doc:​techref:​filesystems#​SquashFS]] ​ |  [[doc:​techref:​filesystems#​JFFS2]] ​ |  //​none// ​ | ^ filesystem ​  ​| ​ //​none// ​                   |  //​none// ​                    ​| ​ [[doc:​techref:​filesystems#​SquashFS]] ​ |  [[doc:​techref:​filesystems#​JFFS2]] ​ |  //​none// ​ |
 +
 +Here is a LibreOffice Calc ODS for better understanding:​ [[https://​web.archive.org/​web/​20131021013058/​http://​ubuntuone.com/​2aPBH9pwkxtYzy93S0cS1z]].
  
 ==== Partitions ==== ==== Partitions ====
 Since the partitions are nested we look at this whole thing in layers: Since the partitions are nested we look at this whole thing in layers:
   - Layer0: So we have the Flashchip, 8MiB in size, which is soldered to the PCB and connected to the [[doc:​hardware:​SoC]] over [[wp>​Serial Peripheral Interface Bus|SPI (Serial Peripheral Interface Bus)]].   - Layer0: So we have the Flashchip, 8MiB in size, which is soldered to the PCB and connected to the [[doc:​hardware:​SoC]] over [[wp>​Serial Peripheral Interface Bus|SPI (Serial Peripheral Interface Bus)]].
-  - Layer1: We "​partition"​ the space into mtd0 for the bootloader, mtd5 for the firmware and, in this case, mtd4 for the ART (Atheros Radio Test) - it contains ​mac addresses and calibration data for the wifi (EEPROM). If it is missing or corrupt, ''​ath9k''​ (wireless driver) won't come up anymore.+  - Layer1: We "​partition"​ the space into mtd0 for the bootloader, mtd5 for the firmware and, in this case, mtd4 for the ART (Atheros Radio Test) - it contains calibration data for the wifi (EEPROM). If it is missing or corrupt, ''​ath9k''​ (wireless driver) won't come up anymore. The bootloader (128k ) contains of the u-boot 64k block AND a data section which contains the MAC,WPS-PIN and type description. If no mac is configured wifi will not work correctly due to a faulty MAC.
   - Layer2: we subdivide mtd5 (firmware) into mtd1 (kernel) and mtd2 (rootfs); In the generation process of the firmware (see [[doc:​howto:​obtain.firmware.generate]]) the Kernel binary file is first packed with [[wp>​Lempel–Ziv–Markov chain algorithm|LZMA]],​ then the obtained file is packed with [[wp>​gzip]] and then this file will be written onto the raw flash (mtd1) without being part of any filesystem!   - Layer2: we subdivide mtd5 (firmware) into mtd1 (kernel) and mtd2 (rootfs); In the generation process of the firmware (see [[doc:​howto:​obtain.firmware.generate]]) the Kernel binary file is first packed with [[wp>​Lempel–Ziv–Markov chain algorithm|LZMA]],​ then the obtained file is packed with [[wp>​gzip]] and then this file will be written onto the raw flash (mtd1) without being part of any filesystem!
   - Layer3: we subdivide rootfs even further into mtd3 for rootfs_data and the rest for an unnamed partition which will accommodate the SquashFS-partition.   - Layer3: we subdivide rootfs even further into mtd3 for rootfs_data and the rest for an unnamed partition which will accommodate the SquashFS-partition.
Line 33: Line 46:
 ==== Mount Points ==== ==== Mount Points ====
   * <color magenta>''/''</​color>​ this is your entire root filesystem, it comprises ''/​rom''​ and ''/​overlay''​. Please ignore ''/​rom''​ and ''/​overlay''​ and use exclusively ''/''​ for your daily routines!   * <color magenta>''/''</​color>​ this is your entire root filesystem, it comprises ''/​rom''​ and ''/​overlay''​. Please ignore ''/​rom''​ and ''/​overlay''​ and use exclusively ''/''​ for your daily routines!
-  * <color magenta>''/​rom''</​color> ​ contains all the basic files, like ''​busybox'',​ ''​dropbear''​ or ''​iptables'' ​including ​default configuration files. ​Does not contain the kernel. ​Files in this directory are on the SqashFS partition, and thus cannot be deleted. But, because we use mini_fo ​filesystem, so called //​overlay-whiteout//​-symlinks can be created on the JFFS2 partition.+  * <color magenta>''/​rom''</​color> ​ contains all the basic files, like ''​busybox'',​ ''​dropbear''​ or ''​iptables''​. It also includes ​default configuration files used when booting into [[doc:​howto:​generic.failsafe|OpenWrt Failsafe mode]]. It does not contain the Linux kernel. ​All files in this directory are located ​on the SqashFS partition, and thus cannot be altered or deleted. But, because we use overlay_fs ​filesystem, so called //​overlay-whiteout//​-symlinks can be created on the JFFS2 partition.
   * <color magenta>''/​overlay''</​color> ​ is the writable part of the file system that gets merged with ''/​rom''​ to create a uniform ''/''​-tree. It contains anything that was written to the router after [[doc:​howto:​generic.flashing|installation]],​ e.g. changed configuration files, additional packages installed with ''​[[doc:​techref:​opkg]]'',​ etc. It is formated with JFFS2.   * <color magenta>''/​overlay''</​color> ​ is the writable part of the file system that gets merged with ''/​rom''​ to create a uniform ''/''​-tree. It contains anything that was written to the router after [[doc:​howto:​generic.flashing|installation]],​ e.g. changed configuration files, additional packages installed with ''​[[doc:​techref:​opkg]]'',​ etc. It is formated with JFFS2.
  
-Rather than deleting ​the filesinsert a whiteout, a special high-priority entry that marks the file as deletedFile system ​code that sees whiteout ​entry for file F behaves ​as if F does not exist.+Whenever ​the system is asked to look for an existing file in ''/''​it first looks in ''/​overlay''​and if not there, then in ''/​rom''​. ​ In this way ''/​overlay''​ overrides ''/​rom''​ and creates the effect of writable ''/''​ while much of the content is safely and efficiently stored in the read-only ''/​rom''​. 
 + 
 +When the system ​is asked to delete a file that is in ''/​rom'',​ it instead creates ​corresponding ​entry in ''/​overlay'',​ a whiteout. ​ A whiteout is a symlink to ''​(overlay-whiteout)''​ that mostly ​behaves ​like a file that doesn'​t ​exist.
  
 <code bash> <code bash>
-#!/bin/bash+#!/bin/sh
 # shows all overlay-whiteout symlinks # shows all overlay-whiteout symlinks
  
-find /overlay -type l | while read FILE +find /overlay -type l -exec sh -c \ 
-  do +    '​for x; do [ "$(readlink -n -- "$x"​)"​ = "​(overlay-whiteout)" ] && printf %s\\n "$x"done' -- {} +
-    ​-z "$FILE" ] && break +
-    if ls -la "$FILE" ​2>&- | grep -q '(overlay-whiteout)'; then +
-    echo "$FILE" +
-    fi +
-  ​done+
 </​code>​ </​code>​
  
-==== Directories ​==== +==== Directory Structure / Filesystem Hierarchy ​====
--> [[doc:​howto:​user.beginner.fhs]] +
 **//''​NOTE1''//:​** If the Kernel was part of the SquashFS, we could not control where exactly on the flash it is written to (on which blocks its data is contained). Thus we could not tell the bootloader to simply load and execute certain blocks on the flash storage, but would have to address it with path and filename. Now this would not be bad, but in order to that the bootloader would have to understand the SquashFS filesystem, which it does not. The embedded bootloaders we utilize with OpenWrt generally have no concept of filesystems,​ thus they cannot address files by path and filename. They pretty much assume that the start of the trx data section is executable code.\\ **//''​NOTE1''//:​** If the Kernel was part of the SquashFS, we could not control where exactly on the flash it is written to (on which blocks its data is contained). Thus we could not tell the bootloader to simply load and execute certain blocks on the flash storage, but would have to address it with path and filename. Now this would not be bad, but in order to that the bootloader would have to understand the SquashFS filesystem, which it does not. The embedded bootloaders we utilize with OpenWrt generally have no concept of filesystems,​ thus they cannot address files by path and filename. They pretty much assume that the start of the trx data section is executable code.\\
 **//''​NOTE2''//:​** The denomination ''"​firmware"''​ usually is used for the entire data on the flash comprising the boot loader and any other data necessary to operate the device, such as ART, NVRAM, FIS, etc, but we also use it to name only the parts that are being rewritten. Don't let this confuse you ;-)\\ **//''​NOTE2''//:​** The denomination ''"​firmware"''​ usually is used for the entire data on the flash comprising the boot loader and any other data necessary to operate the device, such as ART, NVRAM, FIS, etc, but we also use it to name only the parts that are being rewritten. Don't let this confuse you ;-)\\
Line 101: Line 109:
 ^ Atheros ​ |  RedBoot ​ |  firmware ​ |  ''​FIS recovery'' ​ |  ''​RedBoot config'' ​ |  ''​boardconfig'' ​ | ^ Atheros ​ |  RedBoot ​ |  firmware ​ |  ''​FIS recovery'' ​ |  ''​RedBoot config'' ​ |  ''​boardconfig'' ​ |
  
-The partition or partitions containing so called //Special Configuration Data// differ very much from each other. Example: The ''​ART''​-partition you will meet in conjunction with Atheros-Wireless and U-Boot, contains only data regarding the wireless driver, while the ''​NVRAM''​-partition of broadcom devices is used for much more then only that.+The partition or partitions containing so called //Special Configuration Data// differ very much from each other. Example: The ''​ART''​-partition you will meet in conjunction with Atheros-Wireless and U-Boot, contains only data regarding the wireless driver, while the ''​NVRAM''​-partition of broadcom devices is used for much more than only that. There are special utilities to access and modify special configuration partitions. For Broadcom devices this is the ''​nvram''​ utility. To find out what is written in ''​NVRAM''​ you can run ''​nvram show''​. 
 + 
 +Note that clearing these special configuration data partitions like ''​ART,​ NVRAM''​ and ''​FIS''​ does not clear much of OpenWrt'​s configuration,​ unlike other router software which keep configuration data solely in e.g. ''​NVRAM''​. Instead, as a consequence of using the overlay_fs filesystem configuration with JFFS2 flash partition, the whole file system is writable and allows the flexibility of extending your OpenWrt installation in any way you want. OpenWrt'​s main configuration is therefore just kept in the root file system, using [[doc/​uci|UCI]] configuration files. For convenience,​ many other packages are made UCI compatible. If you want to reset your complete installation you should use OpenWrt'​s built-in functionality such as [[doc/​howto/​generic.sysupgrade|sysupgrade]] to restore settings, by clearing the JFFS2 partition. Or, if you cannot boot normally, you can wipe or change the JFFS2 partition using OpenWrt'​s [[doc/​howto/​generic.failsafe|failsafe mode]] (look in your device'​s dedicated page for information how to boot into failsafe).  
 + 
  
 ==== broadcom with CFE ==== ==== broadcom with CFE ====
Line 126: Line 138:
 |  lzma decompress ​ |  lzma compreszsed kernel ​ | |  lzma decompress ​ |  lzma compreszsed kernel ​ |
  
-Now, the boot loader boots into an LZMA program which decompresses the kernel into memory ​and executes it. It adds one second to the bootup time, but it saves a large chunk of flash space. (And if that wasn't amusing enough, it turns out the boot loader does know gzip compression,​ so we gzip compressed the LZMA decompression program)+Now, the boot loader boots into an LZMA program which decompresses the kernel into RAM and executes it. It adds one second to the bootup time, but it saves a large chunk of flash space. (And if that wasn't amusing enough, it turns out the boot loader does know gzip compression,​ so we gzip compressed the LZMA decompression program)
  
 Immediately following the kernel is the filesystem. We use SquashFS for this because it's a highly compressed readonly filesystem -- remember that altering the contents of the trx in any way would invalidate the crc, so we put our writable data in a JFFS2 partition, which is outside the trx. This means that our firmware looks like this: Immediately following the kernel is the filesystem. We use SquashFS for this because it's a highly compressed readonly filesystem -- remember that altering the contents of the trx in any way would invalidate the crc, so we put our writable data in a JFFS2 partition, which is outside the trx. This means that our firmware looks like this:
Line 137: Line 149:
  
 ---- ----
 +
 ===== Explanations ===== ===== Explanations =====
 ==== What is an Image File? ==== ==== What is an Image File? ====
-An image file is byte by byte copy of data contained in a file system. If you installed a Debian or a Windows in the usual way onto one or two harddisc ​partitions and would afterwards copy the whole content byte by byte from the hard disc into one file:+An image file is byte by byte copy of data contained in a file system. If you installed a Debian or a Windows in the usual way onto one or two hard disk partitions and would afterwards copy the whole content byte by byte from the hard disk into one file:
  
 <code bash> <code bash>
Line 152: Line 165:
   * [[doc/​techref/​header]]   * [[doc/​techref/​header]]
   * [[doc/​howto/​obtain.firmware.generate]] If you want to read about the **Image Generator**,​ go back to [[doc:​howto:​obtain.firmware]] and choose the second way.   * [[doc/​howto/​obtain.firmware.generate]] If you want to read about the **Image Generator**,​ go back to [[doc:​howto:​obtain.firmware]] and choose the second way.
-  * About [[http://​skaya.enix.org/​wiki/​FirmwareFormat|Broadcom ​Firware ​Format]]+  * About [[http://​skaya.enix.org/​wiki/​FirmwareFormat|Broadcom ​Firmware ​Format]]
doc/techref/flash.layout.1360167609.txt.bz2 · Last modified: 2013/02/06 17:20 by buntalo