User Tools

Site Tools


doc:howto:mwan3

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
doc:howto:mwan3 [2013/11/02 15:13]
timmillerdyck check ISP servers for reachability
doc:howto:mwan3 [2015/07/01 18:41] (current)
pier4r [12.09, tplink wdr3600, mwan3 - 1.4-24, two wan connections]
Line 4: Line 4:
     * [[https://​forum.openwrt.org/​viewtopic.php?​id=39052|OpenWrt Forum: New package: mwan3; multi-wan policy routing; testers wanted]]; much of the content below comes from forum posts by Adze or Arfett on this thread     * [[https://​forum.openwrt.org/​viewtopic.php?​id=39052|OpenWrt Forum: New package: mwan3; multi-wan policy routing; testers wanted]]; much of the content below comes from forum posts by Adze or Arfett on this thread
     * there is documentation available for policy routing on Linux, e.g. [[http://​www.policyrouting.org/​PolicyRoutingBook/​ONLINE/​TOC.html|Policy Routing With Linux - Online Edition by Matthew G. Marsh]]     * there is documentation available for policy routing on Linux, e.g. [[http://​www.policyrouting.org/​PolicyRoutingBook/​ONLINE/​TOC.html|Policy Routing With Linux - Online Edition by Matthew G. Marsh]]
-    * source code and development versions on github.com: [[https://​github.com/​Adze1502/​mwan]]+    * source code on github.com: [[https://​github.com/​openwrt/​packages/​tree/​master/​net/​mwan3]] 
 +    * source code on github.com: [[https://​github.com/​openwrt/​packages/​tree/​master/​net/​mwan3-luci]] 
 +    * old source code and/or development versions on github.com: [[https://​github.com/​Adze1502/​mwan]]
  
   * Related pages:   * Related pages:
     * ''​multiwan''​ is a different package for managing multiple WAN connections:​ see [[doc:​uci:​multiwan]] and [[doc:​howto:​multiwan.failower]]     * ''​multiwan''​ is a different package for managing multiple WAN connections:​ see [[doc:​uci:​multiwan]] and [[doc:​howto:​multiwan.failower]]
  
-  * Tested on+===== Latest release ===== 
-    A TP-LINK TL-WR1043ND hardware version 1.10 router ​(ar71xx platform) ([[toh/​tp-link/​tl-wr1043nd]]) using OpenWrt 12.09 + 
-    A NetGear WNDR3800 router ​(ar71xx platform) ([[toh/​netgear/​wndr3800]]) using OpenWrt ​12.09 +The mwan3 packages current as of 2015-03-17 are
-    The mwan3 packages current as of 2013-10-14: + 
-      * mwan3_1.2-19_all.ipk +  ​mwan3_1.4-25_all.ipk (for OpenWrt 12.09 "​Attitude Adjustment"​ release only) 
-      * luci-app-mwan3_1.1-13_all.ipk+  mwan3_1.5-10_all.ipk ​(for OpenWrt 14.07 "​Barrier Breaker"​ release only) 
 +  * mwan3_1.6-1_all.ipk ​(for OpenWrt ​15.05 "Chaos Calmer"​ release & trunk only) 
 +  luci-app-mwan3_1.3-5_all.ipk (for OpenWrt 14.07 "​Barrier Breaker"​ release only) 
 +  * luci-app-mwan3_1.4-2_all.ipk (for OpenWrt 15.05 "Chaos Calmer"​ release & trunk only) 
 + 
 +See below for the download and installation procedures.
  
 ===== Description ===== ===== Description =====
Line 27: Line 34:
     * this can be customized based on source IP, destination IP, source port(s), destination port(s), type of IP protocol     * this can be customized based on source IP, destination IP, source port(s), destination port(s), type of IP protocol
   * administration and configuration is through a LuCI configuration module   * administration and configuration is through a LuCI configuration module
-  * up to 15 physical and/or logical WAN interfaces are supported+  * up to 250 physical and/or logical WAN interfaces are supported
  
 ==== Creators ==== ==== Creators ====
Line 40: Line 47:
   * If you have multiple internet connections,​ you want to control which traffic goes through which WANs   * If you have multiple internet connections,​ you want to control which traffic goes through which WANs
   * Mwan3 can handle multiple levels of primary and backup interfaces, load-balanced or not. Different sources can have different primary or backup WANs.   * Mwan3 can handle multiple levels of primary and backup interfaces, load-balanced or not. Different sources can have different primary or backup WANs.
-  * Mwan3 uses flowmask ​to be compatible with other packages (such as OpenVPN, PPTP VPN, QoS-script, Tunnels, etc) as you can configure traffic to use the default routing table.+  * Mwan3 uses netfilter mark mask to be compatible with other packages (such as OpenVPN, PPTP VPN, QoS-script, Tunnels, etc) as you can configure traffic to use the default routing table.
   * Mwan3 can also load-balance traffic originating from the router itself   * Mwan3 can also load-balance traffic originating from the router itself
  
Line 51: Line 58:
 ==== Mwan3 architecture ==== ==== Mwan3 architecture ====
  
-  * Mwan3 is triggered by hotplug-events. When an interface comes up, it creates ​new routing ​tables ​and new iptables rules. A new routing table is created for each interface ​and for each policy. It then sets up iptables rules and uses iptables MARK to mark certain traffic. Based on these rules, the kernel determines which routing table to use. When an interface goes down, mwan3 deletes all the rules and routes to that interface ​in all created routing tables+  * Mwan3 is triggered by hotplug-events. When an interface comes up, it creates ​a custom ​routing ​table and iptables rules. A new routing table is created for each interface. It then sets up iptables rules and uses iptables MARK to mark certain traffic. Based on these rules, the kernel determines which routing table to use. When an interface goes down, mwan3 deletes all the rules and routes to that interface. 
-  * Once all the routes and rules are initially set up, mwan3 exits. The kernel takes care of all the routing decisions. If a new interface hotplug event occurs, mwan3 will run again to adjust ​routing ​tables as needed.+  * Once all the routes and rules are initially set up, mwan3 exits. The kernel takes care of all the routing decisions. If a new interface hotplug event occurs, mwan3 will run again to adjust ​route and tables as needed.
   * A monitoring script (mwan3track) runs in the background checking if each WAN interface is up using a ping test. If an interface goes down, the script issues a hotplug event to cause mwan3 to adjust routing tables to the interface failure.   * A monitoring script (mwan3track) runs in the background checking if each WAN interface is up using a ping test. If an interface goes down, the script issues a hotplug event to cause mwan3 to adjust routing tables to the interface failure.
  
-===== Prerequisites =====+==== Mwan3 routing ​====
  
-==== OpenWrt version ====+The following steps are taken to route a packet with mwan3:
  
-  * OpenWrt 12.09 or later is needed+Every incoming packet (this includes router originated traffic) ​is handled by the iptables mwan3_hook. This hook takes 5 steps:
  
-==== Package dependencies ====+  - Restore mark if previous set. If successful marked, goto step 5. 
 +  - Check if the packet arrives on a wan interface. If originated from a local connected ip network, then mark packet with iface_id 255 (default). If the packet is from another (non-local) network and arrives on wan interface, then mark it with iface_id. If successful marked, goto step 5. 
 +  - Check if packet destined for an ip connected (local) network. If so then mark packet with iface_id 255 (default) and goto step 5. 
 +  - Apply user rules and mark with configured iface_id. If no match leave unmarked. 
 +  - If marked then save mark.
  
-  * The following ​packages ​are required, but they should be installed automatically if missing when mwan3 is installed so there is no need to manually install them beforehand +Remember that iptables only marks the packet, it does not make routing decisions. Next in line are the ip rules. In following ​order they are:
-    * libc, ip, iptables, iptables-mod-conntrack-extra,​ iptables-mod-ipopt,​ kmod-ipt-conntrack-extra,​ kmod-ipt-ipopt+
  
-==== Package conflicts ====+  - Ip rules 1001 till 1250 are for wan interface 1 till 250 respectively. This rule says: If packet is incoming from wan interface use main routing table, regardless of mark. 
 +  - Ip rules 2001 till 2250 are for wan interface 1 till 250 respectively. This rule says: If packet is marked with iface_id [1-250], use the corresponding wan interface routing table. 
 +  - Ip rule 2254 is a blackhole/​unreachable rule. This rule says: If packet is marked with iface_id 254 (unreachable),​ drop packet and return icmp unreachable.
  
-  * Ensure no other multiple WAN package is installed such as ''​multiwan''​ -- having ''​multiwan''​ installed at the same time as mwan3 is known not to work+Next come the routing tables. These are really simple. There is just the standard main routing table and one routing table containing one gateway for each wan interface. Hopes this make troubleshooting easier.
  
-==== Hardware ​====+==== Mwan3 and IPv6 ====
  
-  * A multiple interface router ​is needed. At least three interfaces need to exist for the minimal configuration:​ inside LAN, WAN1 and WAN2The simplest way to do this is use a VLAN-capable router such that individual switch ports can be put into their own VLANs, thus each becoming separate interfaces.+It is ok to have ipv6 and mwan3 running on the same routerOnly ipv6 is ignored by mwan3. Ipv6 routing is done without intervention of any mwan3 rule/route. (Source: Adze's post at https://​forum.openwrt.org/​viewtopic.php?​pid=243603#​p243603)
  
-==== Test external DNS/​mail/​etc. servers for access from each WAN interface ​====+==== Mwan3 command-line options ​====
  
-  * Users in the forum have reported problems with DNS resolution or being unable to send e-mail after implementing WAN load-balancing or failover using mwan3 +There are now some cli commands ​to help you troubleshoot or show status:
-  * The usual cause is they are using the DNS servers or a mail (SMTP/​POP/​IMAP) server provided by the ISP of the wan1 (original WAN) interface and when the router starts sending traffic out the wan2 interface, the ISP blocks access to its servers because the traffic is now coming from an address that is not in their own network. This is a common security configuration by ISPs and has nothing ​to do with mwan3 specifically. +
-  * Option 1Before implementing any multiple WAN configuration,​ test any ISP-provided services to see if they are reachable from "​foreign"​ IP addresses and ensure that they can still be used from source IPs not on the ISPs network. +
-  * Option 2: Change settings to switch to using servers that are known to be accessible from anywhere +
-    * For DNS servers, using Google Public DNS (at IPs 8.8.8.8 and 8.8.4.4) is a good choice+
  
-===== Multiple WAN interface and routing table preparation =====+<​code>​ 
 +root@OpenWrt:​~#​ mwan3 
 +Syntax: /​usr/​sbin/​mwan3 [command]
  
-==== Rename ​the first WAN interface to be "​wan1" ​(optional but recommended====+Available commands: 
 + start Start the service 
 + stop Stop the service 
 + ​restart Restart the service 
 + ​reload Reload configuration files (or restart if that fails) 
 + ​enable Enable service autostart 
 + ​disable Disable service autostart
  
-  * When adding multiple WAN interfaces to a device, leaving the original (first) WAN interface named "​wan"​ is a source of future confusion. + ifup <​iface>​ Start service on interface 
-  * Also, the default mwan3 configuration files assume the first WAN interface ​is named "​wan1"​ and the second WAN interface is named "​wan2"​ + ifdown <​iface>​ Stop service on interface 
-  * For these reasons, it is suggested to rename the original (first) WAN interface ​to be "​wan1"​ before proceeding+ ​interfaces Show interfaces status 
 + ​policies Show policies status 
 + rules Show rules status 
 + ​status Show all status 
 +</​code>​
  
-NoteThe WAN interfaces can be named other names, or be left as "​wan",​ "​wan2",​ etc. mwan3 supports this configuration as well. Also, OpenWrt has a limit as well where WAN interface names need to be short or NATing doesn'​t work. Eight characters works but not much more than that. +  * Example:
- +
-=== SSH ===+
  
 <​code>​ <​code>​
-vi /​etc/​config/​network+root@OpenWrt:​~#​ mwan3 status 
 +Interface status: 
 +Interface wan is online (tracking active) 
 +Interface wan2 is online (tracking active)
 </​code>​ </​code>​
  
-  * change name from '​wan'​ to '​wan1'​+===== Prerequisites =====
  
-<​code>​ +==== OpenWrt version ====
-...+
  
-config interface '​wan1'​ +  * OpenWrt 12.09 or later is needed
-        ​... +
-</​code>​+
  
-<​code>​ +==== Hardware ====
-vi /​etc/​config/​firewall +
-</​code>​+
  
-  * change just the network ​interface ​name from '​wan'​ to '​wan1'​ +Any router configured with multiple WAN interfaces running OpenWrt 12.09 or later should work. Just pick a device with good OpenWrt support, preferably one with VLAN support for the additional ​interface ​flexibility VLAN support provides.
-    * Note: don't change the name of the "​wan"​ firewall zone -- this is different than the "​wan"​ interface name+
  
-<​code>​ +At least three interfaces need to exist for the minimal configuration:​ inside LAN, WAN1 and WAN2The simplest way to do this is use VLANs to put individual switch ports into their own VLANs, thus each becoming separate interfaces.
-...+
  
-config zone +  * As examples, the following specific devices are working well with mwan3: 
-        ​option name             wan +    * A TP-LINK TL-WR1043ND hardware version 1.10 router (ar71xx platform) ([[toh/​tp-link/​tl-wr1043nd]]) using OpenWrt 12.09. 
-        ​option network ​         '​wan1'​ +    * A TP-LINK TL-WR3600 router (ar71xx platform) ([[toh/​tp-link/​tl-wdr3600]]) using OpenWrt 12.09. 
-        ... +    * A openwrt 12.09 mips metarouter over a mikrotik r493g routeros 6.27 ([[inbox/​doc/​mikrotik_metarouter_openwrt]])
-</code>+    * A NetGear WNDR3800 router (ar71xx platform) ([[toh/netgear/​wndr3800]]) using OpenWrt 12.09.
  
-  * Reboot the device+==== Package dependencies ====
  
-<​code>​ +  * The following packages are required, but they should be installed automatically if missing when mwan3 is installed so there is no need to manually install them beforehand 
-reboot +    * libc, ip, iptables, iptables-mod-conntrack-extra,​ iptables-mod-ipopt,​ kmod-ipt-conntrack-extra,​ kmod-ipt-ipopt 
-</code>+ 
 +==== Package conflicts ==== 
 + 
 +  * Ensure no other multiple WAN package is installed such as ''​multiwan''​ -- having ''​multiwan''​ installed at the same time as mwan3 is known not to work 
 + 
 +==== Test external DNS/​mail/​etc. servers for access from each WAN interface ==== 
 + 
 +  * Users in the forum have reported problems with DNS resolution or being unable to send e-mail after implementing WAN load-balancing or failover using mwan3 
 +  * The usual cause is they are using the DNS servers or a mail (SMTP/​POP/​IMAP) server provided by the ISP of the wan (original WAN) interface and when the router starts sending traffic out the wan2 interface, the ISP blocks access to its servers because the traffic is now coming from an address that is not in their own network. This is a common security configuration by ISPs and has nothing to do with mwan3 specifically. 
 +  * Option 1: Before implementing any multiple WAN configuration,​ test any ISP-provided services to see if they are reachable from "​foreign"​ IP addresses and ensure that they can still be used from source IPs not on the ISPs network. 
 +  * Option 2: Change settings to switch to using servers that are known to be accessible from anywhere 
 +    * For DNS servers, using Google Public DNS (at IPs 8.8.8.8 and 8.8.4.4) is a good choice 
 +  * Option 3: Create user rules for traffic destined to "​private"​ DNS servers to only exit the correct interface.  
 + 
 +===== Multiple WAN interface and routing table preparation ===== 
 + 
 +==== The first WAN interface is named "​wan"​ ==== 
 + 
 +The mwan3 default configuration file assumes two WAN interfaces are named "​wan"​ and "​wan2"​. If this is not the case, edit the file /etc/config/mwan3 to configure the "​interface"​ definitions to have the same WAN names as defined in network configuration.
  
 ==== Create and configure a second WAN interface ==== ==== Create and configure a second WAN interface ====
Line 140: Line 172:
   * Go to Network > Interfaces and add a new interface name for the new eth0.x adapter   * Go to Network > Interfaces and add a new interface name for the new eth0.x adapter
     * name the new VLAN physical interface "​wan2"​     * name the new VLAN physical interface "​wan2"​
-    * **don'​t create a bridge over the specified interface**+    * **:!: don't create a bridge over the specified interface ​:!:**
     * configure the new wan2 interface IP details     * configure the new wan2 interface IP details
     * assign the new wan2 interface to the wan firewall zone     * assign the new wan2 interface to the wan firewall zone
Line 146: Line 178:
 Create additional WAN interfaces (e.g. wan3, ...) as desired if more than two WAN connections will be used. Create additional WAN interfaces (e.g. wan3, ...) as desired if more than two WAN connections will be used.
  
-==== Prepare default routing table for WAN interfaces and test ====+==== Prepare ​and the check the default ​OS routing table for WAN interfaces and test ====
  
-  * Before doing anything with mwan3, ensure each WAN interface is working and that the default routing table is correctly configured for multiple WAN connections +  * :!: **IMPORTANT:​** ​ :!: Before doing anything with mwan3, ensure ​that each WAN interface is working and that the default ​OS routing table is correctly configured for multiple WAN connections. Test each interface with a manual ping test before installing mwan3. There have been multiple reports of mwan3 problems on the forum when the problem is actually at the OS level and visible before mwan3 is even installed.
-  * Here are the steps to do this+
  
-=== Configure a different metric for each WAN interface ===+=== Step 1: Configure a different metric for each WAN interface ===
  
-  * mwan3 will set custom routes. ​Instead of the default route metric setting of 0, specifically configure each WAN interface to use a **different** routing metric. This metric only has effect on the default routing table, not on the mwan3 routing tables.+  * Instead of the default route metric setting of 0, specifically configure each WAN interface to use a **different** routing metric. This metric ​will only have an effect on the default routing table, not on the mwan3 routing tables.
   * The default (primary) WAN interface should have the lowest metric (e.g. 10) and each additional WAN interface a higher metric (e.g. 20, 30, etc.)   * The default (primary) WAN interface should have the lowest metric (e.g. 10) and each additional WAN interface a higher metric (e.g. 20, 30, etc.)
-  * Every WAN interface should have "Use default gateway"​ enabled+  * Every WAN interface should have "Use default gateway"​ enabled ​if this option is present
  
 Note: PPPoE connections only show the "Use gateway metric"​ option if "Use default gateway"​ is enabled Note: PPPoE connections only show the "Use gateway metric"​ option if "Use default gateway"​ is enabled
  
-== WAN1 setting ==+== WAN setting ==
  
-WAN1 is the default WAN interface in this example, and so will get the lowest metric of 10.+WAN is the default WAN interface in this example, and so will get the lowest metric of 10.
  
   * Network > Interfaces   * Network > Interfaces
-    * WAN1 > Edit+    * WAN > Edit
       * Advanced Settings       * Advanced Settings
         * Use default gateway: enabled         * Use default gateway: enabled
Line 190: Line 221:
 <​code>​ <​code>​
 Network Target ​   IPv4-Gateway Metric Network Target ​   IPv4-Gateway Metric
-wan1    ​0.0.0.0/0 ...          10+wan     0.0.0.0/0 ...          10
 wan2    0.0.0.0/0 ...          20 wan2    0.0.0.0/0 ...          20
 lan ... lan ...
 ... ...
 </​code>​ </​code>​
 +
 +  * Ensure that every WAN interface has a gateway IP defined and has metric defined
 +
 +=== Troubleshooting ===
 +
 +== Interfaces are missing a metric value ==
 +
 +  * There was a report of some wireless interfaces missing a metric value and a gateway. The mwan3 syslog message error was "​user.warn mwan3: Could not find gateway for interface wan1 (wlan0)"​
 +    * the fix is to add manual static routes -- see the forum thread at [[https://​forum.openwrt.org/​viewtopic.php?​pid=230631#​p230631]] and following
  
 ==== Verify outbound traffic on each WAN interface ==== ==== Verify outbound traffic on each WAN interface ====
  
-Check if above configuration ​works by trying to ping www.google.com from each interface.+Check that each WAN interfaces ​works by trying to ping www.google.com ​out from each interface. Ensure all interfaces are correctly sending and receiving traffic before proceeding.
  
-=== Test WAN1 connection ===+=== Test the wan (first WAN) connection ===
  
-  * WAN1 is hardware interface eth0.1 in this example:+  * wan is hardware interface eth0.1 in this example:
  
 <​code>​ <​code>​
Line 216: Line 256:
   * Ensure the single ping is successful on this interface ("1 packets transmitted,​ 1 packets received, 0% packet loss" should be displayed)   * Ensure the single ping is successful on this interface ("1 packets transmitted,​ 1 packets received, 0% packet loss" should be displayed)
  
-=== Test WAN2 connection ===+=== Test the wan2 connection ===
  
-  * WAN2 is hardware interface eth0.2 in this example:+  * wan2 is hardware interface eth0.2 in this example:
  
 <​code>​ <​code>​
Line 231: Line 271:
  
   * Ensure the single ping is successful on this interface ("1 packets transmitted,​ 1 packets received, 0% packet loss" should be displayed)   * Ensure the single ping is successful on this interface ("1 packets transmitted,​ 1 packets received, 0% packet loss" should be displayed)
 +
 +=== Test all other WAN connections ===
 +
 +  * Repeat as above to ensure every WAN connection that has been created is working
  
 ===== Ensure the CONNTRACK module is enabled in OpenWrt ===== ===== Ensure the CONNTRACK module is enabled in OpenWrt =====
Line 236: Line 280:
 mwan3 requires that the CONNTRACK module is enabled and active on its WAN interfaces. mwan3 requires that the CONNTRACK module is enabled and active on its WAN interfaces.
  
-  * If the interfaces are in the "​wan"​ firewall zone, and the "​Masquerading"​ option is enabled for the firewall zone, the CONNTRACK module is enabled by default already (this is the usual case)+  * If the interfaces are in the "​wan"​ firewall zone, and the "​Masquerading"​ option is enabled for the firewall zone, the CONNTRACK module is enabled by default already (this is the default OpenWrt configuration)
   * If masquerading/​NAT is **not** enabled for the WAN interface (for example, if just routing without NAT is being using between the LAN and your different WAN interfaces),​ you need to add the following rule to the LAN and WAN zone configurations in your /​etc/​config/​firewall:​   * If masquerading/​NAT is **not** enabled for the WAN interface (for example, if just routing without NAT is being using between the LAN and your different WAN interfaces),​ you need to add the following rule to the LAN and WAN zone configurations in your /​etc/​config/​firewall:​
  
Line 246: Line 290:
   * For more information,​ see [[http://​wiki.openwrt.org/​doc/​uci/​firewall#​note.on.connection.tracking.notrack|OpenWRT conntrack/​notrack]]   * For more information,​ see [[http://​wiki.openwrt.org/​doc/​uci/​firewall#​note.on.connection.tracking.notrack|OpenWRT conntrack/​notrack]]
  
-===== Download ​packages =====+===== Manual download of packages =====
  
-The mwan3 packages ​aren'​t ​in the OpenWrt ​standard package ​repository. The two packages need to be separately downloaded ​and installed.+This step is only **required** for OpenWrt 12.09. In OpenWrt 14.07 "​Barrier Breaker"​ and later, the mwan3 packages ​are in the standard package ​repositories ​and no manual download is required.
  
-  * Go to the download site at [[http://​www.mediafire.com/​folder/​fvd0r4i8n4ikg/​mwan3]] ​and check the file names of the latest versions of the mwan3 package and the luci-app-mwan3 package. Download the most recent versions available.+Adze and Arfett keep the OpenWrt ​package ​repositories up to date and you are more likely to download ​the recommended latest "​stable"​ version from there.
  
-There doesn'​t seem to be a way to get a persistent direct download link using the MediaFire.com service, which is needed to download the packages directly to OpenWrt using wgetThe suggested alternative is to download each using a web browser on a PC and then transfer each package using scp.+  * http://213.136.13.52/​mwan3_latest_all.ipk 
 +  * http://​213.136.13.52/​luci-app-mwan3_latest_all.ipk
  
-  ​* Here is a sample ​PuTTY pscp command ​to copy both files from the current directory ​to the /tmp directory ​on the OpenWrt router using SCP (SSH secure copy). Enter the root password for the router when prompted to do so.+The recommended download method involves connecting to your router'​s command line via telnet or SSH and downloading the installation files to your /tmp directory. 
 + 
 +  ​* Here is a sample ​of the router ​command ​line method of downloading ​the files to the /tmp directory ​with the wget program.
  
 <​code>​ <​code>​
-"C:\Program Files (x86)\PuTTY\pscp.exe"​ -scp *.ipk root@192.168.1.1:/tmp+root@OpenWrtcd /tmp 
 +root@OpenWrt:/​tmp#​ rm mwan3_latest_all.ipk 
 +root@OpenWrt:/​tmp#​ wget http://213.136.13.52/​mwan3_latest_all.ipk 
 +root@OpenWrt:/tmp# rm luci-app-mwan3_latest_all.ipk 
 +root@OpenWrt:/​tmp#​ wget http://​213.136.13.52/​luci-app-mwan3_latest_all.ipk
 </​code>​ </​code>​
  
 ===== Installation ===== ===== Installation =====
 +
 +==== OpenWrt 14.07 and later ====
 +
 +  * This is the method when using the mwan3 packages from the standard OpenWrt package repository
 +
 +=== LuCi web interface method ===
 +
 +  * Go to System > Software
 +    * click "​Update lists" to get the latest package databases
 +    * In the "​Download and install package:"​ box, enter "​luci-app-mwan3"​ and click OK to download and install the luci-app-mwan3 package and all related packages, including mwan3 itself and all dependencies
 +
 +=== SSH method ===
 +
 +<​code>​
 +# update package list to prepare for package dependency downloads
 +opkg update
 +
 +# back up the current mwan3 configuration file just in case the automatic backup doesn'​t work
 +cp -a /​etc/​config/​mwan3 /​etc/​config/​mwan3-tempbackup
 +
 +# install luci-app-mwan3,​ mwan3 and all required dependencies
 +opkg install luci-app-mwan3
 +</​code>​
 +
 +==== OpenWrt 12.09 ====
 +
 +  * This is the method when using manually downloaded mwan3 packages (see above)
  
 <​code>​ <​code>​
Line 267: Line 345:
 # update package list to prepare for package dependency downloads # update package list to prepare for package dependency downloads
 opkg update opkg update
 +
 +# back up the current mwan3 configuration file just in case the automatic backup doesn'​t work
 +cp -a /​etc/​config/​mwan3 /​etc/​config/​mwan3-tempbackup
  
 # install mwan3 and all required package dependencies # install mwan3 and all required package dependencies
Line 275: Line 356:
 </​code>​ </​code>​
  
-==== Reboot ​if needed ====+==== Restart LuCI or reboot ​if needed ====
  
-  ​Check that there is new tab in LuCINetwork > MWAN3 Multi-WAN +To ensure the new menu item for mwan3 appears, restart the web server hosting the LuCI interface (or just reboot the router). 
-  * Reboot ​the router if this tab is not present+ 
 +  ​Go to System > Startup 
 +    * click the "​Restart"​ button next to the uhttpd process 
 +    * Re-log into LuCi 
 + 
 +A new menu entry "​Network > Load Balancing"​ should now be present. 
 + 
 +==== Upgrades ==== 
 + 
 +The upgrade path is almost the same as the new install path. 
 + 
 +  * Install mwan3 as per above, ​in the same way as in a new installation 
 +  * The configuration file /​etc/​config/​mwan3 will be the newdefault version. The previous mwan3 file will be renamed as "mwan3-opkg.backup"​ but otherwise left intact in /etc/config as well 
 +  * Manually update ​the new version of /​etc/​config/​mwan3 to ensure interface names are correct and that previously configured interface, member, policy and rule settings are re-entered. Note that some keywords have been dropped over time (e.g. "​option reroute"​) so follow the lead of how the default mwan3 file is configured. 
 +  * Restart mwan3 
 +  * Check its status in LuCI or from the command line (see below) to confirm all expected interfaces are up and testing OK
  
 ===== MWAN3 configuration ===== ===== MWAN3 configuration =====
Line 303: Line 399:
 | ''​up''​ | number | no | ''​5''​ | Number of successful tests to considered link as alive | | ''​up''​ | number | no | ''​5''​ | Number of successful tests to considered link as alive |
 | ''​down''​ | number | no | ''​5''​ | Number of failed tests to considered link as dead | | ''​down''​ | number | no | ''​5''​ | Number of failed tests to considered link as dead |
-| ''​reroute''​ | boolean | no | ''​0''​ | If set to "​1",​ mwan3 rules will apply to traffic sourced from this interface | 
  
 The primary reason to change the default settings is to shorten the time before an interface is failed-over (by reducing the ping interval and number of pings before the interface is down) or lengthen the time to avoid a false link failure report. Please note that if you change the timeout value on low bandwidth interfaces (e.g. 3g) or busy interfaces, that false time-outs can occur. A timeout value of less then 2 seconds is not recommended. The primary reason to change the default settings is to shorten the time before an interface is failed-over (by reducing the ping interval and number of pings before the interface is down) or lengthen the time to avoid a false link failure report. Please note that if you change the timeout value on low bandwidth interfaces (e.g. 3g) or busy interfaces, that false time-outs can occur. A timeout value of less then 2 seconds is not recommended.
Line 309: Line 404:
 A typical interface section looks like this: A typical interface section looks like this:
 <​code>​ <​code>​
-config ​'interface' ​'wan1+config interface 'wan
-        option ​'enabled' ​'​1'​ +        option enabled '​1'​ 
-        list 'track_ip' ​'​8.8.4.4'​ +        list track_ip '​8.8.4.4'​ 
-        list 'track_ip' ​'​8.8.8.8'​ +        list track_ip '​8.8.8.8'​ 
-        list 'track_ip' ​'​208.67.222.222'​ +        list track_ip '​208.67.222.222'​ 
-        list 'track_ip' ​'​208.67.220.220'​ +        list track_ip '​208.67.220.220'​ 
-        option ​'reliability' ​'​2'​ +        option reliability '​2'​ 
-        option ​'count' ​'​1'​ +        option count '​1'​ 
-        option ​'timeout' ​'​2'​ +        option timeout '​2'​ 
-        option ​'interval' ​'​5'​ +        option interval '​5'​ 
-        option ​'down' ​'​3'​ +        option down '​3'​ 
-        option ​'up' ​'8+        option up '​8'​
-        option '​reroute'​ '0'+
 </​code>​ </​code>​
  
-  * **Reroute details:​** +  * The default configuration has wan2 disabled -- enable ​the wan2 interface in the mwan3 configuration
-    * 0: This is the default setting. In this case, traffic originating from this interface ​(such as pinging out from the router) will not be affected by mwan3 rules. If the wan with the lowest metric ​in the default routing tables is dead, traffic from the router itself (with the source IP of this interface) will not go out. Note that routed traffic sourced from other interfaces or sources from lan hosts will be unaffected and handled by mwan3 rules as expected. Even when this is set to "​0"​. +
-    * 1: If set to "​1",​ new outgoing traffic connections originating from this interface (with the source IP of this interface) will be handled by MWAN3 rules. For example, this will allow new connections from the router itself to failover to an alternate WAN interface if the first WAN interface goes down. Set this to "​1"​ for all wan interfaces to have all router originated traffic through all possible outgoing traffic paths controlled by mwan3 rules. +
- +
-  * A working mwan3 config has at least 2 and at most 15 interfaces configured.+
  
 ==== Member configuration ==== ==== Member configuration ====
Line 357: Line 447:
 A typical policy section looks like this: A typical policy section looks like this:
 <​code>​ <​code>​
-config ​'policy' ​'​wan1_wan2_loadbalanced+config policy 'balanced
- list 'use_member' ​'​wan1_m1_w3+        list use_member 'wan_m1_w3
- list 'use_member' ​'​wan2_m1_w2'​+        list use_member '​wan2_m1_w2'​
 </​code>​ </​code>​
  
   * If a policy is not referenced by a specific traffic rule, the policy will not do anything, so it is fine to leave unused policies in place in case they are desired in the future.   * If a policy is not referenced by a specific traffic rule, the policy will not do anything, so it is fine to leave unused policies in place in case they are desired in the future.
  
-  * If you have a traffic rule that matches a policy, but all the members (interfaces) for that policy are down, it will not match any mwan3 ip rule. Therefore, it will use the main routing table to determine which interface to useIf you don't want this traffic to leave certain interfacesyou have to add some firewall rules.+  * If you have a traffic rule that matches a policy, but all the members (interfaces) for that policy are down, the exit strategy for that policy defaults ​to "​unreachable"​This is configurable with the last_resort option. Valid values are: blackholeunreachable or default.
  
-  * A working mwan3 config has at least 1 and at most 84 policies ​configured.+  * A working mwan3 config has at least 1 policy ​configured. 
 + 
 +  * **Ensure no policy name is longer than 15 characters**
  
 ==== Rule configuration ==== ==== Rule configuration ====
Line 374: Line 466:
 ^ Name ^ Type ^ Required ^ Default ^ Description ^ ^ Name ^ Type ^ Required ^ Default ^ Description ^
 | ''​use_policy''​ | string | yes | //(none)// | Use this policy for traffic that matches or set to ''​default''​ to use the default routing table to lookup | | ''​use_policy''​ | string | yes | //(none)// | Use this policy for traffic that matches or set to ''​default''​ to use the default routing table to lookup |
-| ''​equalize''​ | boolean | no | ''​0''​ | If set to "​0",​ routing lookup cache is used, which can result in new sessions to the same destination get routed over the same wan interface. If set to "​1",​ for each new connection a new wan interface is selected, but no cache is used | 
 | ''​src_ip''​ | ip address | no | any | Match traffic from the specified source ip address | | ''​src_ip''​ | ip address | no | any | Match traffic from the specified source ip address |
 | ''​src_port''​ | port or range | no | any | Match traffic from the specified source port or port range, if relevant ''​proto''​ is specified | | ''​src_port''​ | port or range | no | any | Match traffic from the specified source port or port range, if relevant ''​proto''​ is specified |
Line 380: Line 471:
 | ''​dest_ip''​ | ip address | no | any | Match traffic directed to the specified destination ip address | | ''​dest_ip''​ | ip address | no | any | Match traffic directed to the specified destination ip address |
 | ''​dest_port''​ | port or range | no | any | Match traffic directed at the given destination port or port range, if relevant ''​proto''​ is specified | | ''​dest_port''​ | port or range | no | any | Match traffic directed at the given destination port or port range, if relevant ''​proto''​ is specified |
 +| ''​ipset''​ | string | no | //(none)// | Match traffic directed at the given destination ip address to an ipset set |
 +| ''​sticky''​ | boolean | no | 0 | Allow traffic from the same source ip address within the timeout limit to use same wan interface as prior session |
 +| ''​timeout''​ | number | no | 600 | Stickiness timeout value in seconds |
  
-* There are a number of sample rules defined to show how they work. Edit the rules as desired and delete all the rest of the default rules.+  ​* There are a number of sample rules defined to show how they work. Edit the rules as desired and delete all the rest of the default rules
 +  * The options ipset, sticky and timeout are only available in version 1.6 or higher.
  
 A typical rule section looks like this: A typical rule section looks like this:
 <​code>​ <​code>​
-config ​'rule'​ +config rule '​default_rule
- option ​'dest_ip' ​'88.154.0.0/16+        option dest_ip '0.0.0.0/0
- option ​'​proto'​ '​tcp'​ +        option use_policy 'wan_wan2_wan3'
- option '​dest_port'​ '​1024:​65535'​ +
- option '​equalize'​ '​1'​ +
- option 'use_policy' ​'​wan1_wan2_loadbalanced'+
 </​code>​ </​code>​
 +
 +=== Stickiness and ipset ===
 +
 +Mwan3 version 1.6 has sticky and ipset support. Stickiness lets you route new session over the same wan interface as the previous session, as long as the time between the new and the previous session is shorter then the timeout value (default 600s). This can solve some problems with https sites, which don't allow a new source address within the same cookie/​https session. Ipset lets you route traffic over wan interfaces based on set of ip addresses. A set can be created by hand, by dnsmasq based on domain names, or your own script. Mwan3 rules with ipset option will try to match destination ip address to the configured ipset.
 +
 +<​code>​
 +config rule '​youtube'​
 +    option sticky ‘1'
 +    option timeout ‘300'​
 +    option ipset '​youtube'​
 +    option dest_port '​80,​443'​
 +    option proto '​tcp'​
 +    option use_policy '​balanced'​
 +</​code>​
 +
 +With sticky set to 1, this rule has now sticky enabled. When a packet for a new session matches this rule, its source ip address and interface mark are stored in an ipmark set with a timeout of 300 seconds (default 600). When packet for a second new session from the same lan host within the timeout period matches this rule, it will use the same wan interface as the first packet and the timeout counter is reset back to 300 again.
 +
 +**Stickiness is on a per rule basis. With this example, all traffic from lan hosts will use the same wan interface for all youtube hosts, even if the source or destination ip address differs.**
 +
 +The option ipset matches only destination ip addresses. This example will only work if your lan clients use the dnsmasq server as their one and only dns server. Mwan3 will create the ipset set for you if it does not exist already. For this to work you need to configure a rule in your /​etc/​dnsmasq.conf file:
 +
 +<​code>​
 +ipset=/​youtube.com/​youtube
 +</​code>​
 +
  
   * **Order is important.** Rules are evaluated in top-to-bottom order, with the first matching rule applying. The rule name is just descriptive and has no operational impact. If no match is found, routing lookup is done via the default routing table. ​   * **Order is important.** Rules are evaluated in top-to-bottom order, with the first matching rule applying. The rule name is just descriptive and has no operational impact. If no match is found, routing lookup is done via the default routing table. ​
  
   * A working mwan3 config has at least 1 rule configured.   * A working mwan3 config has at least 1 rule configured.
- 
 ==== Example configuration ==== ==== Example configuration ====
  
  <​code>​  <​code>​
-config ​'interface' ​'wan1+config interface 'wan
- option ​'enabled' ​'​1'​ +        option enabled '​1'​ 
- list 'track_ip' ​'​8.8.4.4'​ +        list track_ip '​8.8.4.4'​ 
- list 'track_ip' ​'​8.8.8.8'​ +        list track_ip '​8.8.8.8'​ 
- list 'track_ip' ​'​208.67.222.222'​ +        list track_ip '​208.67.222.222'​ 
- list 'track_ip' ​'​208.67.220.220'​ +        list track_ip '​208.67.220.220'​ 
- option ​'reliability' ​'​2'​ +        option reliability '​2'​ 
- option ​'count' ​'​1'​ +        option count '​1'​ 
- option ​'timeout' ​'​2'​ +        option timeout '​2'​ 
- option ​'interval' ​'​5'​ +        option interval '​5'​ 
- option ​'down' ​'​3'​ +        option down '​3'​ 
- option ​'up' ​'8+        option up '​8'​
- option '​reroute'​ '0'+
  
-config ​'interface' ​'​wan2'​ +config interface '​wan2'​ 
- option ​'enabled' ​'​1'​ +        option enabled '​1'​ 
- list 'track_ip' ​'​8.8.8.8'​ +        list track_ip '​8.8.8.8'​ 
- list 'track_ip' ​'​208.67.220.220'​ +        list track_ip '​208.67.220.220'​ 
- option ​'reliability' ​'​1'​ +        option reliability '​1'​ 
- option ​'count' ​'​1'​ +        option count '​1'​ 
- option ​'timeout' ​'​2'​ +        option timeout '​2'​ 
- option ​'interval' ​'​5'​ +        option interval '​5'​ 
- option ​'down' ​'​3'​ +        option down '​3'​ 
- option ​'up' ​'8+        option up '​8'​
- option '​reroute'​ '0'+
  
-config ​'member' ​'​wan1_m1_w3+config member 'wan_m1_w3
- option ​'interface' ​'wan1+        option interface 'wan
- option ​'metric' ​'​1'​ +        option metric '​1'​ 
- option ​'weight' ​'​3'​+        option weight '​3'​
  
-config ​'member' ​'​wan2_m1_w2+config member 'wan_m2_w3
- option ​'interface' ​'wan2+        option interface 'wan
- option ​'metric' ​'1+        option metric '2
- option ​'weight' ​'2'+        option weight '3'
  
-config 'policy' 'wan1_wan2_loadbalanced+config ​member ​'wan2_m1_w2' 
- list 'use_member'​ '​wan1_m1_w3+        option interface ​'wan2
- list 'use_member'​ '​wan2_m1_w2'+        ​option metric ​'1
 +        ​option weight ​'2'
  
-config 'rule+config ​member ​'wan2_m2_w2
- option 'dest_ip' '​0.0.0.0/​0'​ +        option ​interface ​'wan2' 
- option '​use_policy'​ 'wan1_wan2_loadbalanced'+        option metric '​2'​ 
 +        option weight '​2'​ 
 + 
 +config policy '​wan_only'​ 
 +        list use_member '​wan_m1_w3'​ 
 + 
 +config policy '​wan2_only'​ 
 +        list use_member '​wan2_m1_w2'​ 
 + 
 +config policy '​balanced'​ 
 +        list use_member '​wan_m1_w3'​ 
 +        list use_member '​wan2_m1_w2'​ 
 + 
 +config policy '​wan_wan2'​ 
 +        list use_member '​wan_m1_w3'​ 
 +        list use_member '​wan2_m2_w2'​ 
 + 
 +config policy '​wan2_wan'​ 
 +        list use_member '​wan_m2_w3'​ 
 +        list use_member '​wan2_m1_w2'​ 
 + 
 +config rule '​sticky_even'​ 
 +        option src_ip ​'​0.0.0.0/​0.0.0.1
 +        option ​dest_port ​'443' 
 +        option proto '​tcp'​ 
 +        option ​use_policy 'wan_wan2'​ 
 + 
 +config rule '​sticky_odd'​ 
 +        option src_ip '​0.0.0.1/​0.0.0.1'​ 
 +        option dest_port '​443'​ 
 +        option proto '​tcp'​ 
 +        option use_policy '​wan2_wan'​ 
 + 
 +config rule '​default_rule'​ 
 +        option dest_ip '​0.0.0.0/​0'​ 
 +        option use_policy ​'balanced'
 </​code>​ </​code>​
  
 ===== Further configuration tips ===== ===== Further configuration tips =====
  
-==== OpenWrt hotplug script fix ====+==== OpenWrt hotplug script fix (OpenWrt 12.09 only) ==== 
 + 
 +**This is for OpenWrt 12.09 only. The OpenWrt 14.07 hotplug scripts were substantially re-written and there is no evidence yet that the workaround below is needed on OpenWrt 14.07.**
  
   * Forum member tcherenato found that adding a 1 second pause to the OpenWrt hotplug launch script helps prevent occasional segmentation faults when mwan3 performs hotplug operations. It is not known currently what the root issue is (or even if it is in mwan3 at all) but the change is recommended.   * Forum member tcherenato found that adding a 1 second pause to the OpenWrt hotplug launch script helps prevent occasional segmentation faults when mwan3 performs hotplug operations. It is not known currently what the root issue is (or even if it is in mwan3 at all) but the change is recommended.
Line 472: Line 624:
 ===== Start mwan3 ===== ===== Start mwan3 =====
  
-  * mwan3 automatically will start after each reboot but if a reboot has not occurred yet, the package can be manually started.+  * Mwan3 automatically will start after each reboot but if a reboot has not occurred yet, the package can be manually started.
   * see the "​Administration"​ section below   * see the "​Administration"​ section below
  
Line 480: Line 632:
  
 ===== Verification of basic operation ===== ===== Verification of basic operation =====
 +
 +==== Check MWAN3 status in cli ====
 +
 +<​code>​
 +root@OpenWrt:​~#​ mwan3 status
 +Interface status:
 +Interface wan is online (tracking active)
 +Interface wan2 is online (tracking active)
 +
 +Policy balanced:
 + wan2 (40%)
 + wan (60%)
 +
 +Policy wan1_only:
 + wan (100%)
 +
 +Policy wan2_only:
 + wan2 (100%)
 +
 +Policy wan2_wan:
 + wan2 (100%)
 +
 +Policy wan_wan2:
 + wan (100%)
 +
 +Local connected networks:
 +destination ​       policy ​            ​hits ​    
 +------------------------------------------------
 +127.0.0.0/​8 ​       default ​           22       
 +224.0.0.0/​3 ​       default ​           0        ​
 +192.168.1.0/​24 ​    ​default ​           0        ​
 +192.168.33.0/​24 ​   default ​           0        ​
 +213.154.232.8/​29 ​  ​default ​           0        ​
 +
 +Active rules:
 +source ​            ​destination ​       proto  src-port ​     dest-port ​    ​policy ​         hits     
 +---------------------------------------------------------------------------------------------------
 +0.0.0.0/​0 ​         213.136.223.128/​25 tcp    0:​65535 ​      ​80 ​           wan_wan2 ​       0        ​
 +1.2.3.4 ​           5.6.7.8 ​           udp    12345:​54321 ​  ​12345:​54321 ​  ​wan2_wan ​       0        ​
 +0.0.0.0/​0 ​         0.0.0.0/​0 ​         all                                balanced ​       2862     
 +</​code>​
  
 ==== Check status in the MWAN3 overview page ==== ==== Check status in the MWAN3 overview page ====
  
-  * Network > MWAN3 Multi-WAN+  * Network > Load Balancing
     * Overview     * Overview
       * MWAN3 Multi-WAN Interface Live Status       * MWAN3 Multi-WAN Interface Live Status
Line 492: Line 685:
 ==== Check kernel routing tables ==== ==== Check kernel routing tables ====
  
-  * "ip route show table 0" should show route tables with table numbers 1000 or higher (e.g. 1018, 1020) -- these tables are generated by mwan3 +  * "ip route show table x" ​(where x is interface ID) should show a routing ​table specifically for that interface ​-- these tables are generated by mwan3
-  * a specific table number can be viewed as desired+
  
 ===== Verification of WAN interface load-balancing ===== ===== Verification of WAN interface load-balancing =====
Line 507: Line 699:
 ==== Test interface failover ==== ==== Test interface failover ====
  
-  * Go to Network > MWAN3 Multi-WAN ​> Overview+  * Go to Network > Load Balancing ​> Overview
     * Manually disconnect a WAN connection     * Manually disconnect a WAN connection
     * Wait for interface failure detection to happen -- the mwan3 status display should update     * Wait for interface failure detection to happen -- the mwan3 status display should update
Line 530: Line 722:
 === LuCI === === LuCI ===
  
-  * System ​Startup +  * Network ​Load Balancing > Advanced > Diagnostics 
-    * mwan3 > click "​Stop" ​in the "​Stop"​ column ​to stop the service+    * MWAN Service Control ​> click "​Stop ​MWAN" to stop the service
  
 === SSH === === SSH ===
  
 <​code>​ <​code>​
-/​etc/​init.d/​mwan3 stop+mwan3 stop
 </​code>​ </​code>​
  
Line 545: Line 737:
 === LuCI === === LuCI ===
  
-  * System ​Startup +  * Network ​Load Balancing > Advanced > Diagnostics 
-    * mwan3 > click "​Start" ​in the "​Start"​ column ​to start the service+    * MWAN Service Control ​> click "​Start ​MWAN" to start the service
  
 === SSH === === SSH ===
  
 <​code>​ <​code>​
-/​etc/​init.d/​mwan3 start+mwan3 start
 </​code>​ </​code>​
  
Line 558: Line 750:
 ==== Manual status check ==== ==== Manual status check ====
  
-  * Network > MWAN3 Multi-WAN +  * Network > Load Balancing 
-    * Overview > MWAN3 Multi-WAN ​Interface Live Status +    * Overview > MWAN Interface Live Status 
-      * verify all interfaces show "ONLINE" status+      * verify all interfaces show "Online" status
  
 ==== Automated status check ==== ==== Automated status check ====
Line 575: Line 767:
 It would be good for mwan3 to send some kind of alert (e.g. e-mail, SNMP trap) if it detects a failed interface and performs a failover, or if it performs a fail-back. It would be good for mwan3 to send some kind of alert (e.g. e-mail, SNMP trap) if it detects a failed interface and performs a failover, or if it performs a fail-back.
  
-  * FIXME +If you install the luci-app-mwan3 (mwan3-luci) package it comes with /​etc/​hotplug.d/​iface/​16-mwancustom hotplug shell script which may be modified to perform custom actions on interface ifup/ifdown events. It is already filled with a basic template for gathering information on the current state of WAN connections and just needs your changes in the send_alert() function as well as uncommenting the case statement at the bottom. The mwan3 LuCI package provides a page for editing this hotplug shell script or you may edit via command line.
 ===== Controlling the mapping between internal IP sources and external IPs and interfaces ===== ===== Controlling the mapping between internal IP sources and external IPs and interfaces =====
  
Line 668: Line 859:
  
   * Configure ddns-scripts to use the "​web"​ update mechanism as this will reflect the current active external IP   * Configure ddns-scripts to use the "​web"​ update mechanism as this will reflect the current active external IP
-  * Ensure the mwan3 "​reroute"​ setting is set to 1 (enabled) for all WAN interfaces for this to work as ddns-scripts traffic is originating from the router itself and needs the reroute setting enabled so its outgoing can be redirected by mwan3 failover rules 
  
 === Example 2: Register the external IP of a specific WAN interface using the "​interface"​ source === === Example 2: Register the external IP of a specific WAN interface using the "​interface"​ source ===
Line 702: Line 892:
     * [[doc/​howto/​vpn.openvpn]]     * [[doc/​howto/​vpn.openvpn]]
  
-=== Example 1: Have OpenVPN Server be accessible through multiple WAN interfaces ===+=== Possible problems === 
 +If the openwrt system is an openvpn client 
 +and a zone '​vpn'​ is defined on the vpn interface 
 +and this zone has the masquerading active, for 
 +reasons (yet) unknown the traffic from the internal lan 
 +to the vpn will be able to reach the destination and 
 +go back to the router but then will be not dispatched back to the 
 +lan clients. Disabling mwan3, instead, let the traffic be dispatched 
 +properly. 
 + 
 +It could be a misconfiguration,​ more testing is needed. 
 + 
 +=== Example 1: Have OpenVPN Server be accessible through multiple WAN interfaces ​(server mode) ===
  
 If load-balancing between multiple WAN interfaces, it is desirable to have OpenVPN clients be able to connect through all active WAN interfaces. If load-balancing between multiple WAN interfaces, it is desirable to have OpenVPN clients be able to connect through all active WAN interfaces.
Line 749: Line 951:
   * If load-balancing between multiple active WAN interfaces, the suggested approach is to register multiple DNS A records for the same DNS name. Clients will use just one of the IPs. As per the OpenVPN man page description of the --remote client parameter, "If host is a DNS name which resolves to multiple IP addresses, one will be randomly chosen, providing a sort of basic load-balancing and failover capability."​   * If load-balancing between multiple active WAN interfaces, the suggested approach is to register multiple DNS A records for the same DNS name. Clients will use just one of the IPs. As per the OpenVPN man page description of the --remote client parameter, "If host is a DNS name which resolves to multiple IP addresses, one will be randomly chosen, providing a sort of basic load-balancing and failover capability."​
   * If failing over from a primary to a secondary WAN interface, one approach is to use ddns-scripts to update the IP of the DNS name used by OpenVPN clients   * If failing over from a primary to a secondary WAN interface, one approach is to use ddns-scripts to update the IP of the DNS name used by OpenVPN clients
 +
 +=== Example 2: Use OpenVPN tunnels as virtual wan(s) (client mode) ===
 +
 +If you want to use your OpenVPN client tunnels as virtual wan interfaces in mwan3, you have to make sure that you set a default route with different metric for each tunnel interface. Also most commercial VPN solutions push two static routes to override the standard default gateway. In most cases you don't want this override when using OpenVPN client tunnels in conjunction with mwan3.
 +
 +As a solution you can add the following lines to your OpenVPN client config:
 +
 +<​code>​
 +route-nopull
 +route 0.0.0.0 0.0.0.0 vpn_gateway 20
 +</​code>​
 +
 +This example will ignore the routes pushed from the OpenVPN server and will add a default route with metric 20 over the OpenVPN tunnel interface.
  
 ==== privoxy transparent HTTP proxy ==== ==== privoxy transparent HTTP proxy ====
Line 804: Line 1019:
 The policy "​wan1_wan2_loadbalanced"​ is just an example. Change it to whatever policy you like. The policy "​wan1_wan2_loadbalanced"​ is just an example. Change it to whatever policy you like.
  
-Note that you have to set the reroute option to "​1"​ to have privoxy originated traffic also be handled by mwan3, or else it will always leave the gateway with the lowest metric in the default routing table. 
  
 +==== nodogsplash ====
 +
 +Out of the box, mwan3 does not work with nodogsplash. The problem is that both mwan3 and nodogsplash use the same iptables mark bits. A common symptom of this is the nodogsplash splash page appearing for every page even as an authenticated client. ​
 +
 +However, it is possible to fix this with a minor change! Fortunately,​ it is simple to change the mark bits that nodogsplash uses. Simply add the following lines to '/​etc/​nodogsplash/​nodogsplash.conf'​ to override the marking bits.
 +
 +<​code>​
 +# Change the default marking flags to work with mwan3 and qos-scripts
 +FW_MARK_AUTHENTICATED 262144
 +FW_MARK_TRUSTED 131072
 +FW_MARK_BLOCKED 65536
 +</​code>​
 +
 +These values let nodogsplash work together with mwan3 and also work with standard Openwrt qos-scipts.
 +
 +
 +
 +===== Usage reports =====
 +==== 12.09, tplink wdr3600, mwan3 - 1.4-24, two wan connections ====
 +Premise: very nice piece of work given to the internet community, congrats to the contributors.
 +We, users, can only give back a bit of experience and documentation,​ still somehow useful.
 +
 +In a simulated test environment described as follows:
 +<​file>​
 +external network ​ <---> (ip a.b.c.118) router1 (ip 192.0.2.1) ​ <---> (ip 192.0.2.166 - wan) tplink (ip 192.168.1.1) ​ <---> (ip 192.168.1.50) pcA
 +                  <---> (ip a.b.c.224) router2 (ip 192.0.10.1) <---> (ip 192.0.10.166 - wan2)       (ip 192.168.10.1) <---> (ip 192.168.10.101)pcB
 +</​file>​
 +
 +The mwan3 was installed on the tplink with the following configuration (apart from wan and wan2) in ''/​etc/​config/​mwan3'':​
 +<​file>​
 +...lines....
 +
 +config member '​wan_m10'​
 +        option interface '​wan'​
 +        option metric '​10'​
 +
 +config member '​wan_m20'​
 +        option interface '​wan'​
 +        option metric '​20'​
 +
 +config member '​wan2_m10'​
 +        option interface '​wan2'​
 +        option metric '​10'​
 +
 +config member '​wan2_m20'​
 +        option interface '​wan2'​
 +        option metric '​20'​
 +
 +config policy '​wan_wan2'​
 +        list use_member '​wan_m10'​
 +        list use_member '​wan2_m20'​
 +
 +config policy '​wan2_wan'​
 +        list use_member '​wan_m20'​
 +        list use_member '​wan2_m10'​
 +
 +config rule '​rule1'​
 +        list comment ​   'from 192.168.1.50 to wan_wan2'​
 +        option src_ip ​  '​192.168.1.0/​24'​
 +        option dest_ip ​ '​0.0.0.0/​0'​
 +        option use_policy '​wan_wan2'​
 +
 +config rule '​rule2'​
 +        list comment ​   'from 192.168.10.101 to wan2_wan'​
 +        option src_ip ​  '​192.168.10.0/​24'​
 +        option dest_ip ​ '​0.0.0.0/​0'​
 +        option use_policy '​wan2_wan'​
 +</​file>​
 +
 +=== Using different wan connections from different Pc-s ===
 +Then with the pcA and pcB two different download were started, and every pc used a different connection. Great.
 +
 +=== Line fail ===
 +When one wan connection was physically disconnected,​ one pc lost the tcp active connections,​
 +but after '​restarting'​ them no problem, the pc was using the other wan connection configured as
 +line backup. Great.
 +
 +=== Line recovered ===
 +If a wan line recovers (let's say A), then the pc that was using the other wan line (configured as backup, let'S say B) is
 +switched back to the wan line A, and this cause another disruption of tcp connections.
 +
 +//Edit: On recovery, connections already established over backup links will not be terminated and continue to traverse over backup wan. Only new connections will be routed over preferred wan.//
 +
 +=== Incoming connections routed behind the router ===
 +Rdp connections to the pc behind the tplink are stable, this means that a service behind the tplink with
 +mwan3 is reachable in a reliable way.
 +
 +Therefore does not happen that an external request coming on one wan connection gets the replies
 +through the other wan connection (at least for failover policies)
 +
 +=== Incoming connections to services on the router ===
 +At least testing with ssh, does not happen that one connection through a wan line is router on the other wan line 
 +in case of line failover. Therefore ssh is stable.
 +
 +=== mwan3 status hints ===
 +''​mwan3 status''​ shows often '​hits'​ only for the last rule if this one is generic (source 0.0.0.0/24 dest 0.0.0.0/24 )
 +like for web traffic. This is a bit misleading if in reality the traffic was
 +split by other rules. I do not know why iptables reports this (mwan3 status just reports what iptables
 +says) but analyzing it with iftop and bmon there is better to asses if the traffic is
 +directed properly using the wan connections.
 +==== Reliable public ip addresses to ping ====
 +After some weeks of continous pinging to opendns Ips (208.67.222.222) and (208.67.220.220),​ seems that opendns does not like this and interrupt the icmp replies for a while, therefore the mwan3 thinks that the lines are down. Could be useful to collect stable internet ip addresses? (possibly from backbone providers).
 +
 +Some possible choices:
 +<​file>​
 +# Level3 communications (large network carrier)
 +4.2.2.2
 +
 +# google dnses
 +8.8.4.4
 +8.8.8.8
 +
 +# facebook.com
 +173.252.120.6
 +
 +# Opendns (with limits after a certain amount of days?)
 +208.67.220.220
 +208.67.222.222
 +
 +</​file>​
 +
 +=== Possible work arounds to test connection not only by ping ? ===
 +Could be possible to implement other ways to test connections,​ maybe through testing ip/ports (with additional packages, of course), like: ''​netcat -z -w 2 208.67.222.222 53 ; echo $?''​
doc/howto/mwan3.1383401602.txt.bz2 · Last modified: 2013/11/02 15:13 by timmillerdyck