User Tools

Site Tools


doc:howto:obtain.firmware.generate

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
doc:howto:obtain.firmware.generate [2013/03/24 22:32]
hcc23 Added how to build the Image Generator from the sources
doc:howto:obtain.firmware.generate [2016/05/14 10:04] (current)
BarTender [Usage]
Line 1: Line 1:
 +====== Image Generator (Image Builder) - EN ======
 +| -> go back to [[doc:​howto:​obtain.firmware]] |
  
 +If you do not want to [[obtain.firmware.download|download]] a prebuilt image or go through the entire [[build|compilation]] process, the alternative is to use **Image Generator** (formerly called **Image Builder**). This is a pre-compiled OpenWrt build environment suitable for creating custom images without the need for compiling.
 +
 +:!: Image Generator will not run on 32bit platform such as i686. You need 64bit linux such as amd64, see [[https://​dev.openwrt.org/​ticket/​17117|ticket]]. You can however run amd64 in VM (i.e. virtualbox) even from 32bit windows.
 +
 +Reasons for using **Image Generator** are:
 +  * Embedding packages directly into the SquashFS to reduce space requirements on the target
 +  * Preconfigure images by embedding packages and configuration files directly into SquashFS, and save manpower when flashing many devices
 +  * Building minimal images. for example without the web interface
 +  * Learning
 +
 +----
 +**Image Generator** is the program that creates the OpenWrt firmware image file. In the process of [[build|compiling OpenWrt]], **Image Generator** is coercively created (compiled), because it is needed to eventually create the image file. It is located in ''/​openwrt/​trunk/​xxx''​ and you can use it, to create more image-files from the packages you obtained during compilation.
 +
 +But you can also simply download an archive which contains **Image Generator** and a whole load of packages from OpenWrt. This archive containes binaries running on amd64 linux.
 +----
 +
 +===== Download OpenWrt Chaos Calmer 15.05.1 =====
 +[[https://​downloads.openwrt.org/​chaos_calmer/​15.05.1/​]]
 +  * Click on your architecture,​ for example //ar71xx//
 +  * Download the ImageBuilder archive.
 +  * Extract the archive and change into the ImageBuilder directory
 +Example: Chaos Calmer for ar71xx architecture
 +<​WRAP>​
 +<​code>​
 +cd ~
 +mkdir openwrt && cd openwrt
 +wget https://​downloads.openwrt.org/​chaos_calmer/​15.05.1/​ar71xx/​generic/​OpenWrt-ImageBuilder-15.05.1-ar71xx-generic.Linux-x86_64.tar.bz2
 +tar -xvjf OpenWrt-ImageBuilder-15.05.1-ar71xx-generic.Linux-x86_64.tar.bz2
 +cd OpenWrt-ImageBuilder-15.05.1-ar71xx-generic.Linux-x86_64
 +</​code>​
 +</​WRAP>​
 +
 +===== Configure Package Repositories =====
 +The **Image Generator** has to be configured for downloading missing packages from official or custom repositories. Otherwise the image generation fails with an error of a missing package. The package sources are configured in the ''​repositories.conf''​ file in the toplevel directory. Sources are specified in //opkg// native config format. This can be either the official package repositories or custom generated repositories.
 +
 +**EXAMPLE:​** To use all official package repositories for ar71xx the repositories.conf should look like:
 +<​WRAP>​
 +<​code>​
 +## Place your custom repositories here, they must match the architecture and version.
 +# src/gz chaos_calmer http://​downloads.openwrt.org/​chaos_calmer/​15.05.1/​ar71xx/​generic/​packages
 +# src custom file:///​usr/​src/​openwrt/​bin/​ar71xx/​packages
 +src/gz chaos_calmer_base http://​downloads.openwrt.org/​chaos_calmer/​15.05.1/​ar71xx/​generic/​packages/​base
 +src/gz chaos_calmer_luci http://​downloads.openwrt.org/​chaos_calmer/​15.05.1/​ar71xx/​generic/​packages/​luci
 +src/gz chaos_calmer_management http://​downloads.openwrt.org/​chaos_calmer/​15.05.1/​ar71xx/​generic/​packages/​management
 +src/gz chaos_calmer_packages http://​downloads.openwrt.org/​chaos_calmer/​15.05.1/​ar71xx/​generic/​packages/​packages
 +src/gz chaos_calmer_routing http://​downloads.openwrt.org/​chaos_calmer/​15.05.1/​ar71xx/​generic/​packages/​routing
 +src/gz chaos_calmer_telephony http://​downloads.openwrt.org/​chaos_calmer/​15.05.1/​ar71xx/​generic/​packages/​telephony
 +## This is the local package repository, do not remove!
 +src imagebuilder file:​packages
 +</​code>​
 +</​WRAP>​
 +
 +===== Usage =====
 +The ''​make image''​ command will create a minimal image with just essential packages included, the behavior can be controlled with three variables passed as arguments:
 +
 +  * //''​PROFILE''//​ - specifies the target image to build
 +  * //''​PACKAGES''//​ - a list of packages to embed into the image
 +  * //''​FILES''//​ - directory with custom files to include
 +
 +Dependencies for "​make"​
 +
 +
 +<​WRAP>​Debian/​Ubuntu
 +<​code>​
 +apt-get install subversion build-essential libncurses5-dev zlib1g-dev gawk git ccache gettext libssl-dev xsltproc wget
 +</​code>​
 +</​WRAP>​
 +
 +<​WRAP>​CentOS/​RHEL
 +<​code>​
 +yum install subversion git gawk gettext ncurses-devel zlib-devel openssl-devel libxslt wget
 +yum group install "​Development Tools"
 +</​code>​
 +</​WRAP>​
 +
 +Syntax:
 +
 +<​WRAP>​
 +<​code>​
 +$ make image PROFILE=XXX PACKAGES="​pkg1 pkg2 pkg3 -pkg4 -pkg5 -pkg6" FILES=files/​
 +</​code>​
 +</​WRAP>​
 +
 +See the sections below for an explanation. After the make command is finished, the generated images are stored in the bin///​architecture//​ directory.
 +
 +|{{:​meta:​icons:​tango:​dialog-information.png?​nolink&​24}}|The example below shows the contents of ''​bin/''​ directory after successfully generating an OpenWrt Barrier Breaker image for WNDR3700. The architecture for WNDR3700 is ar71xx, so the exact location of the image is ''​bin/​ar71xx''​|
 +<​WRAP>​
 +<​code>​
 +user@linux:​~/​openwrt/​OpenWrt-ImageBuilder-ar71xx_generic-for-linux-x86_64$ ls -aR bin
 +bin:
 +.  ..  ar71xx
 +
 +bin/ar71xx:
 +.                                         ​openwrt-ar71xx-generic-vmlinux-lzma.elf
 +..                                        openwrt-ar71xx-generic-wndr3700-squashfs-factory.img
 +md5sums ​                                  ​openwrt-ar71xx-generic-wndr3700-squashfs-factory-NA.img
 +openwrt-ar71xx-generic-rootfs.tar.gz ​     openwrt-ar71xx-generic-wndr3700-squashfs-sysupgrade.bin
 +openwrt-ar71xx-generic-root.squashfs ​     openwrt-ar71xx-generic-wndr3700v2-squashfs-factory.img
 +openwrt-ar71xx-generic-root.squashfs-64k ​ openwrt-ar71xx-generic-wndr3700v2-squashfs-sysupgrade.bin
 +openwrt-ar71xx-generic-uImage-gzip.bin ​   openwrt-ar71xx-generic-wndr3800-squashfs-factory.img
 +openwrt-ar71xx-generic-uImage-lzma.bin ​   openwrt-ar71xx-generic-wndr3800-squashfs-sysupgrade.bin
 +openwrt-ar71xx-generic-vmlinux.bin ​       openwrt-ar71xx-generic-wndrmac-squashfs-factory.img
 +openwrt-ar71xx-generic-vmlinux.elf ​       openwrt-ar71xx-generic-wndrmac-squashfs-sysupgrade.bin
 +openwrt-ar71xx-generic-vmlinux.gz ​        ​openwrt-ar71xx-generic-wndrmacv2-squashfs-factory.img
 +openwrt-ar71xx-generic-vmlinux.lzma ​      ​openwrt-ar71xx-generic-wndrmacv2-squashfs-sysupgrade.bin
 +</​code>​
 +</​WRAP>​
 +==== PROFILE Variable =====
 +Syntax:
 +<​WRAP>​
 +<​code>​
 +$ make image PROFILE=NAMEOFPROFILE
 +</​code>​
 +</​WRAP>​
 +
 +=== Pre-defined Profiles ===
 +Run ''​make info''​ to obtain a list of defined profiles.
 +Example output from ''​make info''​ is listed below.
 +
 +== brcm47xx Profiles ==
 +[[http://​pastebin.com/​5L7Pzq7N|brcm47xx Image Generator Profiles]]
 +
 +== ar71xx Profiles ==
 +  * [[http://​pastebin.com/​WbudpBDJ|ar71xx Image Generator Profiles]]
 +  * [[https://​dev.openwrt.org/​browser/​trunk/​target/​linux/​ar71xx/​files/​arch/​mips/​ath79/​machtypes.h|list of ar71xx profiles in latest trunk]]
 +
 +
 +After you find the appropriate profile pass it to the ''​make image''​ command:
 +
 +**EXAMPLE:​** for WNDR3700 (Atheros ar71xx architecture).
 +
 +|{{:​meta:​icons:​tango:​dialog-information.png?​nolink&​24}}|This will build OpenWrt for WNDR3700 with a bare-minumum feature set due to ''​PACKAGES''​ variable not being defined. For information on how to include or exclude packages, see [[obtain.firmware.generate#​PACKAGES Variable]] |
 +<​WRAP>​
 +<​code>​
 +$ make image PROFILE=WNDR3700
 +</​code>​
 +</​WRAP>​
 +
 +Then you would look for the image in the ''​OpenWrt-ImageBuilder-ar71xx_generic-for-linux-i486/​bin/''​ directory.
 +
 +=== Adding/​Modifying Profiles ===
 +
 +|{{:​meta:​icons:​tango:​48px-emblem-important.svg.png?​nolink&​24}}|Note that at least for the ar71xx architecture the image generation is tied to the profile names. If you add a new profile without also adding an appropriate macro to the image Makefile, no suitable firmware file will get generated when using the custom profile. |
 +|{{:​meta:​icons:​tango:​48px-emblem-important.svg.png?​nolink&​24}}|Make sure to remove the /tmp directory to get modified package selection from profiles to work. |
 +
 +The location of the profiles for the pre-compiled package for //​brcm47xx-for-Linux-i686//​ was //​target/​linux/​brcm47xx/​profiles///​
 +
 +Remarkably, all that needs to be done to add a new profile, is to add a new file to the //​profiles//​ directory.
 +
 +Here is what the //​profiles/​100-Broadcom-b43.mk//​ profile file looks like:  ​
 +
 +<​WRAP>​
 +<​code>​
 +define Profile/​Broadcom-b43
 +  NAME:​=Broadcom BCM43xx WiFi (default)
 +  PACKAGES:​=kmod-b43 kmod-b43legacy
 +endef
 +
 +define Profile/​Broadcom-b43/​Description
 + Package set compatible with hardware using Broadcom BCM43xx cards
 +endef
 +$(eval $(call Profile,​Broadcom-b43))
 +</​code>​
 +</​WRAP>​
 +
 +==== PACKAGES Variable ====
 +The ''​PACKAGES''​ variable specifies a list of packages to include and/or exclude when building an image with Image Generator.
 +
 +Syntax:
 +
 +**NOTE:** If the package name is prefixed with ''​-''​ then the corresponding package will be excluded from being built during ''​make image''​.
 +
 +The example below will include pkg1, pkg2, pkg3, and exclude pkg4, pkg5, pkg6.
 +<​WRAP>​
 +<​code>​
 +$ make image PROFILE=XXX PACKAGES="​pkg1 pkg2 pkg3 -pkg4 -pkg5 -pkg6"
 +</​code>​
 +</​WRAP>​
 +|{{:​meta:​icons:​tango:​48px-emblem-important.svg.png?​nolink&​24}}|**Backfire 10.03.1-rc5 or later**: Image Generator now uses ''​opkg''​ on the host to determine packages to install and to resolve their dependencies (which is especially useful if you use [[#External Repositories]]),​ so the following warning does not apply! |
 +
 +|{{:​meta:​icons:​tango:​48px-dialog-warning.svg.png?​nolink&​24}}|**Backfire 10.03 to 10.03.1-rc4**:​ Using ''​PACKAGES=''​ will not install any of the default packages in your PROFILE. ​ Be sure you include **libuci**. ​ Without it uci won't work.  Your router will boot but none of the interfaces will be configured **effectively bricking your router** (setting ''​boot_wait''​ first will let you recover via tftp). |
 +
 +**Tip:** The list of currently installed packages on your router running OpenWrt can be obtained with the command below:
 +
 +<​WRAP>​
 +<​code>​
 +echo $(opkg list_installed | awk '{ print $1 }')
 +</​code>​
 +</​WRAP>​
 +
 +**Tip:** The list of packages present in the current release of OpenWrt, Barrier Breaker 14.07, can be obtained with this Linux command:
 +
 +**NOTE:** Example is for ar71xx. Change the architecture and name of the config file as necessary.
 +<​WRAP>​
 +<​code>​
 +echo $(wget -qO - http://​downloads.openwrt.org/​barrier_breaker/​14.07/​ar71xx/​generic/​config.ar71xx_generic | sed -ne '​s/​^CONFIG_PACKAGE_\([a-z0-9-]*\)=y/​\1/​ip'​)
 +</​code>​
 +</​WRAP>​
 +
 +**Tip:** If you want to remove some of the default packages (for e.g. you do not need pppoe related packages), consult with the DEFAULT_PACKAGES and DEFAULT_PACKAGES.router section in the include/​target.mk file.
 +
 +**Tip:** External packages compatible with a given build (if you cannot use images from http://​downloads.openwrt.org/​ because you have compiled from trunk and trunk has changed since you compiled your image) can be found in ''​packages''​ directory.
 +
 +==== FILES Variable ====
 +The ''​FILES''​ variable allows custom configuration files to be included in images built with Image Generator. This is especially useful if you need to change the network configuration from default before flashing.
 +
 +Syntax:
 +<​WRAP>​
 +<​code>​
 +$ make image PROFILE=XXX PACKAGES="​XXX"​ FILES=files/​
 +</​code>​
 +</​WRAP>​
 +
 +The following example shows:
 +
 +  -Creating the directory for the configuration files
 +  -Using ''​scp''​ to transfer ''​uci''​ configuration files from an OpenWrt router to the ''​files/​etc/​config''​ directory
 +  -Generating an image for WL500GP with custom packages and ''​uci''​ configuration files
 +
 +
 +<​WRAP>​
 +<​code>​
 +mkdir -p files/​etc/​config
 +scp root@192.168.1.1:/​etc/​config/​network files/​etc/​config/​
 +scp root@192.168.1.1:/​etc/​config/​wireless files/​etc/​config/​
 +scp root@192.168.1.1:/​etc/​config/​firewall files/​etc/​config/​
 +make image PROFILE=WL500GP PACKAGES="​nano openvpn -ppp -ppp-mod-pppoe"​ FILES=files/​
 +</​code>​
 +</​WRAP>​
 +
 +**Note:** The ''​files/''​ folder must be in the same folder where you issue the make command.
 +
 +===== Cleanup =====
 +To clean up temporary build files and generated images, use the ''​make clean''​ command.
 +
 +===== Remove useless files from firmware =====
 +1. Create file '​files_remove'​ with full filenames:
 +<​code>​
 +/​lib/​modules/​3.10.49/​ts_bm.ko
 +/​lib/​modules/​3.10.49/​nf_nat_ftp.ko
 +/​lib/​modules/​3.10.49/​nf_nat_irc.ko
 +/​lib/​modules/​3.10.49/​nf_nat_tftp.ko
 +</​code>​
 +
 +2. Patch Makefile
 +<code bash>
 + ifneq ($(USER_FILES),​)
 + ​ $(MAKE) copy_files
 + endif
 ++
 ++ifneq ($(FILES_REMOVE),​)
 ++ @echo
 ++ @echo Remove useless files
 ++
 ++ while read filename; do \
 ++     rm -rfv "​$(TARGET_DIR)$$filename";​ \
 ++ done < $(FILES_REMOVE);​
 ++endif
 ++
 + ​ $(MAKE) package_postinst
 + ​ $(MAKE) build_image
 +</​code>​
 +
 +3. Rebuild firmware
 +<​code>​
 +# make image \
 +    PROFILE=TLWR841 \
 +    PACKAGES="​igmpproxy ip iptraf kmod-ipt-nathelper-extra openvpn-polarssl tcpdump-mini -firewall -ip6tables -kmod-ip6tables -kmod-ipv6 -odhcp6c -ppp -ppp-mod-pppoe"​ \
 +    FILES_REMOVE="​files_remove"​
 +</​code>​
 +
 +
 +===== Building the Image Generator =====
 +It is possible to build the Image Generator from the sources from scratch:
 +
 +<​WRAP>​
 +<code bash>
 +:​~/​openwrt/​trunk>​ svn up
 +:​~/​openwrt/​trunk>​ rm .config
 +:​~/​openwrt/​trunk>​ make menuconfig
 +</​code>​
 +</​WRAP>​
 +
 +In the graphical configuration,​ select ''​Global Build Settings -> Select all packages by default'',​ exit and save.
 +Then build the image, including ''​IGNORE_ERRORS=1''​ as there might be unmaintained packages that fail to compile.
 +
 +<​WRAP>​
 +<​code>​
 +:​~/​openwrt/​trunk>​ make IGNORE_ERRORS=1
 +</​code>​
 +</​WRAP>​
 +
 +**Note:** Don't call ''​make defconfig''​ or leave an old ''​.config''​ file in the path as ''​Select all packages by default''​ will only set the package selection to ''​[m]''​ for packages that are not already configured otherwise!(''​make defconfig''​ will set most packages to ''​[n]'',​ i.e. //do not build//.)