Differences

This shows you the differences between two versions of the page.

doc:howto:mwan3 [2013/11/02 15:13]
timmillerdyck check ISP servers for reachability
doc:howto:mwan3 [2014/04/17 00:54] (current)
timmillerdyck Updates for newer mwan3 release
Line 12: Line 12:
    * A TP-LINK TL-WR1043ND hardware version 1.10 router (ar71xx platform) ([[toh/tp-link/tl-wr1043nd]]) using OpenWrt 12.09     * A TP-LINK TL-WR1043ND hardware version 1.10 router (ar71xx platform) ([[toh/tp-link/tl-wr1043nd]]) using OpenWrt 12.09
    * A NetGear WNDR3800 router (ar71xx platform) ([[toh/netgear/wndr3800]]) using OpenWrt 12.09     * A NetGear WNDR3800 router (ar71xx platform) ([[toh/netgear/wndr3800]]) using OpenWrt 12.09
-    * The mwan3 packages current as of 2013-10-14+    * The mwan3 packages current as of 2014-04-16
-      * mwan3_1.2-19_all.ipk +      * mwan3_1.4-13_all.ipk 
-      * luci-app-mwan3_1.1-13_all.ipk+      * luci-app-mwan3_1.2-12_all.ipk
===== Description ===== ===== Description =====
Line 27: Line 27:
    * this can be customized based on source IP, destination IP, source port(s), destination port(s), type of IP protocol     * this can be customized based on source IP, destination IP, source port(s), destination port(s), type of IP protocol
  * administration and configuration is through a LuCI configuration module   * administration and configuration is through a LuCI configuration module
-  * up to 15 physical and/or logical WAN interfaces are supported 
==== Creators ==== ==== Creators ====
Line 40: Line 39:
  * If you have multiple internet connections, you want to control which traffic goes through which WANs   * If you have multiple internet connections, you want to control which traffic goes through which WANs
  * Mwan3 can handle multiple levels of primary and backup interfaces, load-balanced or not. Different sources can have different primary or backup WANs.   * Mwan3 can handle multiple levels of primary and backup interfaces, load-balanced or not. Different sources can have different primary or backup WANs.
-  * Mwan3 uses flowmask to be compatible with other packages (such as OpenVPN, PPTP VPN, QoS-script, Tunnels, etc) as you can configure traffic to use the default routing table.+  * Mwan3 uses netfilter mark mask to be compatible with other packages (such as OpenVPN, PPTP VPN, QoS-script, Tunnels, etc) as you can configure traffic to use the default routing table.
  * Mwan3 can also load-balance traffic originating from the router itself   * Mwan3 can also load-balance traffic originating from the router itself
Line 51: Line 50:
==== Mwan3 architecture ==== ==== Mwan3 architecture ====
-  * Mwan3 is triggered by hotplug-events. When an interface comes up, it creates new routing tables and new iptables rules. A new routing table is created for each interface and for each policy. It then sets up iptables rules and uses iptables MARK to mark certain traffic. Based on these rules, the kernel determines which routing table to use. When an interface goes down, mwan3 deletes all the rules and routes to that interface in all created routing tables+  * Mwan3 is triggered by hotplug-events. When an interface comes up, it creates a custom routing table and iptables rules. A new routing table is created for each interface. It then sets up iptables rules and uses iptables MARK to mark certain traffic. Based on these rules, the kernel determines which routing table to use. When an interface goes down, mwan3 deletes all the rules and routes to that interface. 
-  * Once all the routes and rules are initially set up, mwan3 exits. The kernel takes care of all the routing decisions. If a new interface hotplug event occurs, mwan3 will run again to adjust routing tables as needed.+  * Once all the routes and rules are initially set up, mwan3 exits. The kernel takes care of all the routing decisions. If a new interface hotplug event occurs, mwan3 will run again to adjust route and tables as needed.
  * A monitoring script (mwan3track) runs in the background checking if each WAN interface is up using a ping test. If an interface goes down, the script issues a hotplug event to cause mwan3 to adjust routing tables to the interface failure.   * A monitoring script (mwan3track) runs in the background checking if each WAN interface is up using a ping test. If an interface goes down, the script issues a hotplug event to cause mwan3 to adjust routing tables to the interface failure.
 +
 +==== Mwan3 routing ====
 +
 +The following steps are taken to route a packet with mwan3:
 +
 +Every incoming packet (this includes router originated traffic) is handled by the iptables mwan3_hook. This hook takes 5 steps:
 +
 +1. Restore mark if previous set. If successful marked, goto step 5.
 +2. (iptables -L mwan3_ifaces -t mangle -v -n) Check if the packet arrives on a wan interface, but not originated from a local connected ip network. If so then mark packet with iface_id. If the packet is from a ip connected network and arrives on wan interface, then mark it with iface_id 255 (default). If successful marked, goto step 5.
 +3. (iptables -L mwan3_connected -t mangle -v -n) Check if packet destined for a ip connected network. If so then mark packet with 255 (default) and goto step 5.
 +4. (iptables -L mwan3_rules -t mangle -v -n) Apply user rules and mark with configured iface_id. If no match leave unmarked.
 +5. If marked then save mark.
 +
 +Remember that iptables only marks the packet, it does not make routing decisions. Next in line are the ip rules. In following order they are:
 +
 +1. Ip rules 1001 till 1253 are for wan interface 1 till 253 respectively. This rule says: If packet is incoming from wan interface use main routing table, regardless of mark.
 +2. Ip rules 2001 till 2253 are for wan interface 1 till 253 respectively. This rule says: If packet is marked with iface_id (1-253), use the corresponding wan interface routing table.
 +3. Ip rule 2255 is a blackhole/unreachable rule. This rule says: If packet is marked with iface_id 254 (unreachable), drop packet and return icmp unreachable.
 +
 +Next come the routing tables. These are really simple. There is just the standard main routing table and one routing table containing one gateway for each wan interface. Hopes this make troubleshooting easier.
 +
 +==== Mwan3 command-line options ====
 +
 +There are now some cli commands to help you troubleshoot or show status:
 +
 +<code>
 +root@OpenWrt:~# mwan3
 +Syntax: /usr/sbin/mwan3 [command]
 +
 +Available commands:
 + start Start the service
 + stop Stop the service
 + restart Restart the service
 + reload Reload configuration files (or restart if that fails)
 + enable Enable service autostart
 + disable Disable service autostart
 +
 + ifup <iface> Start service on interface
 + ifdown <iface> Stop service on interface
 + interfaces Show interfaces status
 + policies Show policies status
 + rules Show rules status
 + status Show all status
 +</code>
 +
 +  * Example:
 +
 +<code>
 +root@OpenWrt:~# mwan3 status
 +Interface status:
 +Interface wan is online (tracking active)
 +Interface wan2 is online (tracking active)
 +</code>
===== Prerequisites ===== ===== Prerequisites =====
Line 84: Line 136:
===== Multiple WAN interface and routing table preparation ===== ===== Multiple WAN interface and routing table preparation =====
-==== Rename the first WAN interface to be "wan1" (optional but recommended) ====+==== Ensure the first WAN interface is named "wan" ====
-  * When adding multiple WAN interfaces to a device, leaving the original (first) WAN interface named "wan" is a source of future confusion. +The mwan3 default configuration file assumes two WAN interfaces are named "wan" and "wan2". If this is not the case, edit the file /etc/config/mwan3 to configure the "interface" definitions to have the same WAN names as defined in LuCI.
-  * Also, the default mwan3 configuration files assume the first WAN interface is named "wan1" and the second WAN interface is named "wan2" +
-  * For these reasons, it is suggested to rename the original (first) WAN interface to be "wan1" before proceeding +
- +
-Note: The WAN interfaces can be named other names, or be left as "wan", "wan2", etc. mwan3 supports this configuration as well. Also, OpenWrt has a limit as well where WAN interface names need to be short or NATing doesn't work. Eight characters works but not much more than that. +
- +
-=== SSH === +
- +
-<code> +
-vi /etc/config/network +
-</code> +
- +
-  * change name from 'wan' to 'wan1' +
- +
-<code> +
-... +
- +
-config interface 'wan1' +
-        ... +
-</code> +
- +
-<code> +
-vi /etc/config/firewall +
-</code> +
- +
-  * change just the network interface name from 'wan' to 'wan1' +
-    * Note: don't change the name of the "wan" firewall zone -- this is different than the "wan" interface name +
- +
-<code> +
-... +
- +
-config zone +
-        option name            wan +
-        option network          'wan1' +
-        ... +
-</code> +
- +
-  * Reboot the device +
- +
-<code> +
-reboot +
-</code>+
==== Create and configure a second WAN interface ==== ==== Create and configure a second WAN interface ====
Line 159: Line 170:
Note: PPPoE connections only show the "Use gateway metric" option if "Use default gateway" is enabled Note: PPPoE connections only show the "Use gateway metric" option if "Use default gateway" is enabled
-== WAN1 setting ==+== WAN setting ==
-WAN1 is the default WAN interface in this example, and so will get the lowest metric of 10.+WAN is the default WAN interface in this example, and so will get the lowest metric of 10.
  * Network > Interfaces   * Network > Interfaces
-    * WAN1 > Edit+    * WAN > Edit
      * Advanced Settings       * Advanced Settings
        * Use default gateway: enabled         * Use default gateway: enabled
Line 190: Line 201:
<code> <code>
Network Target    IPv4-Gateway Metric Network Target    IPv4-Gateway Metric
-wan1    0.0.0.0/0 ...          10+wan    0.0.0.0/0 ...          10
wan2    0.0.0.0/0 ...          20 wan2    0.0.0.0/0 ...          20
lan ... lan ...
Line 200: Line 211:
Check if above configuration works by trying to ping www.google.com from each interface. Check if above configuration works by trying to ping www.google.com from each interface.
-=== Test WAN1 connection ===+=== Test the wan connection ===
-  * WAN1 is hardware interface eth0.1 in this example:+  * wan is hardware interface eth0.1 in this example:
<code> <code>
Line 216: Line 227:
  * Ensure the single ping is successful on this interface ("1 packets transmitted, 1 packets received, 0% packet loss" should be displayed)   * Ensure the single ping is successful on this interface ("1 packets transmitted, 1 packets received, 0% packet loss" should be displayed)
-=== Test WAN2 connection ===+=== Test the wan2 connection ===
-  * WAN2 is hardware interface eth0.2 in this example:+  * wan2 is hardware interface eth0.2 in this example:
<code> <code>
Line 303: Line 314:
| ''up'' | number | no | ''5'' | Number of successful tests to considered link as alive | | ''up'' | number | no | ''5'' | Number of successful tests to considered link as alive |
| ''down'' | number | no | ''5'' | Number of failed tests to considered link as dead | | ''down'' | number | no | ''5'' | Number of failed tests to considered link as dead |
-| ''reroute'' | boolean | no | ''0'' | If set to "1", mwan3 rules will apply to traffic sourced from this interface | 
The primary reason to change the default settings is to shorten the time before an interface is failed-over (by reducing the ping interval and number of pings before the interface is down) or lengthen the time to avoid a false link failure report. Please note that if you change the timeout value on low bandwidth interfaces (e.g. 3g) or busy interfaces, that false time-outs can occur. A timeout value of less then 2 seconds is not recommended. The primary reason to change the default settings is to shorten the time before an interface is failed-over (by reducing the ping interval and number of pings before the interface is down) or lengthen the time to avoid a false link failure report. Please note that if you change the timeout value on low bandwidth interfaces (e.g. 3g) or busy interfaces, that false time-outs can occur. A timeout value of less then 2 seconds is not recommended.
Line 309: Line 319:
A typical interface section looks like this: A typical interface section looks like this:
<code> <code>
-config 'interface' 'wan1+config interface 'wan
-        option 'enabled' '1' +        option enabled '1' 
-        list 'track_ip' '8.8.4.4' +        list track_ip '8.8.4.4' 
-        list 'track_ip' '8.8.8.8' +        list track_ip '8.8.8.8' 
-        list 'track_ip' '208.67.222.222' +        list track_ip '208.67.222.222' 
-        list 'track_ip' '208.67.220.220' +        list track_ip '208.67.220.220' 
-        option 'reliability' '2' +        option reliability '2' 
-        option 'count' '1' +        option count '1' 
-        option 'timeout' '2' +        option timeout '2' 
-        option 'interval' '5' +        option interval '5' 
-        option 'down' '3' +        option down '3' 
-        option 'up' '8+        option up '8'
-        option 'reroute' '0'+
</code> </code>
-  * **Reroute details:** +  * The default configuration has wan2 disabled -- enable the wan2 interface in the mwan3 configuration
-    * 0: This is the default setting. In this case, traffic originating from this interface (such as pinging out from the router) will not be affected by mwan3 rules. If the wan with the lowest metric in the default routing tables is dead, traffic from the router itself (with the source IP of this interface) will not go out. Note that routed traffic sourced from other interfaces or sources from lan hosts will be unaffected and handled by mwan3 rules as expected. Even when this is set to "0". +
-    * 1: If set to "1", new outgoing traffic connections originating from this interface (with the source IP of this interface) will be handled by MWAN3 rules. For example, this will allow new connections from the router itself to failover to an alternate WAN interface if the first WAN interface goes down. Set this to "1" for all wan interfaces to have all router originated traffic through all possible outgoing traffic paths controlled by mwan3 rules. +
- +
-  * A working mwan3 config has at least 2 and at most 15 interfaces configured.+
==== Member configuration ==== ==== Member configuration ====
Line 357: Line 362:
A typical policy section looks like this: A typical policy section looks like this:
<code> <code>
-config 'policy' 'wan1_wan2_loadbalanced+config policy 'balanced
- list 'use_member' 'wan1_m1_w3+       list use_member 'wan_m1_w3
- list 'use_member' 'wan2_m1_w2'+       list use_member 'wan2_m1_w2'
</code> </code>
Line 366: Line 371:
  * If you have a traffic rule that matches a policy, but all the members (interfaces) for that policy are down, it will not match any mwan3 ip rule. Therefore, it will use the main routing table to determine which interface to use. If you don't want this traffic to leave certain interfaces, you have to add some firewall rules.   * If you have a traffic rule that matches a policy, but all the members (interfaces) for that policy are down, it will not match any mwan3 ip rule. Therefore, it will use the main routing table to determine which interface to use. If you don't want this traffic to leave certain interfaces, you have to add some firewall rules.
-  * A working mwan3 config has at least 1 and at most 84 policies configured.+  * A working mwan3 config has at least 1 policy configured.
==== Rule configuration ==== ==== Rule configuration ====
Line 374: Line 379:
^ Name ^ Type ^ Required ^ Default ^ Description ^ ^ Name ^ Type ^ Required ^ Default ^ Description ^
| ''use_policy'' | string | yes | //(none)// | Use this policy for traffic that matches or set to ''default'' to use the default routing table to lookup | | ''use_policy'' | string | yes | //(none)// | Use this policy for traffic that matches or set to ''default'' to use the default routing table to lookup |
-| ''equalize'' | boolean | no | ''0'' | If set to "0", routing lookup cache is used, which can result in new sessions to the same destination get routed over the same wan interface. If set to "1", for each new connection a new wan interface is selected, but no cache is used | 
| ''src_ip'' | ip address | no | any | Match traffic from the specified source ip address | | ''src_ip'' | ip address | no | any | Match traffic from the specified source ip address |
| ''src_port'' | port or range | no | any | Match traffic from the specified source port or port range, if relevant ''proto'' is specified | | ''src_port'' | port or range | no | any | Match traffic from the specified source port or port range, if relevant ''proto'' is specified |
Line 385: Line 389:
A typical rule section looks like this: A typical rule section looks like this:
<code> <code>
-config 'rule' +config rule 'default_rule
- option 'dest_ip' '88.154.0.0/16+       option dest_ip '0.0.0.0/0
- option 'proto' 'tcp' +       option use_policy 'wan_wan2_wan3'
- option 'dest_port' '1024:65535' +
- option 'equalize' '1' +
- option 'use_policy' 'wan1_wan2_loadbalanced'+
</code> </code>
Line 400: Line 401:
<code> <code>
-config 'interface' 'wan1+config interface 'wan
- option 'enabled' '1' +       option enabled '1' 
- list 'track_ip' '8.8.4.4' +       list track_ip '8.8.4.4' 
- list 'track_ip' '8.8.8.8' +       list track_ip '8.8.8.8' 
- list 'track_ip' '208.67.222.222' +       list track_ip '208.67.222.222' 
- list 'track_ip' '208.67.220.220' +       list track_ip '208.67.220.220' 
- option 'reliability' '2' +       option reliability '2' 
- option 'count' '1' +       option count '1' 
- option 'timeout' '2' +       option timeout '2' 
- option 'interval' '5' +       option interval '5' 
- option 'down' '3' +       option down '3' 
- option 'up' '8+       option up '8'
- option 'reroute' '0'+
-config 'interface' 'wan2' +config interface 'wan2' 
- option 'enabled' '1' +       option enabled '1' 
- list 'track_ip' '8.8.8.8' +       list track_ip '8.8.8.8' 
- list 'track_ip' '208.67.220.220' +       list track_ip '208.67.220.220' 
- option 'reliability' '1' +       option reliability '1' 
- option 'count' '1' +       option count '1' 
- option 'timeout' '2' +       option timeout '2' 
- option 'interval' '5' +       option interval '5' 
- option 'down' '3' +       option down '3' 
- option 'up' '8+       option up '8'
- option 'reroute' '0'+
-config 'member' 'wan1_m1_w3+config member 'wan_m1_w3
- option 'interface' 'wan1+       option interface 'wan
- option 'metric' '1' +       option metric '1' 
- option 'weight' '3'+       option weight '3'
-config 'member' 'wan2_m1_w2+config member 'wan_m2_w3' 
- option 'interface' 'wan2' +        option interface 'wan
- option 'metric' '1' +       option metric '2' 
- option 'weight' '2'+        option weight '3' 
 + 
 +config member 'wan2_m1_w2' 
 +        option interface 'wan2' 
 +       option metric '1' 
 +        option weight '2' 
 + 
 +config member 'wan2_m2_w2' 
 +        option interface 'wan2' 
 +        option metric '2' 
 +        option weight '2' 
 + 
 +config policy 'wan_only' 
 +        list use_member 'wan_m1_w3' 
 + 
 +config policy 'wan2_only' 
 +        list use_member 'wan2_m1_w2' 
 + 
 +config policy 'balanced' 
 +        list use_member 'wan_m1_w3' 
 +        list use_member 'wan2_m1_w2' 
 + 
 +config policy 'wan_wan2' 
 +        list use_member 'wan_m1_w3' 
 +        list use_member 'wan2_m2_w2' 
 + 
 +config policy 'wan2_wan' 
 +        list use_member 'wan_m2_w3' 
 +        list use_member 'wan2_m1_w2' 
 + 
 +config rule 'sticky_even' 
 +        option src_ip '0.0.0.0/0.0.0.1' 
 +       option dest_port '443' 
 +        option proto 'tcp' 
 +        option use_policy 'wan_wan2'
-config 'policy' 'wan1_wan2_loadbalanced+config rule 'sticky_odd' 
- list 'use_member' 'wan1_m1_w3+        option src_ip '0.0.0.1/0.0.0.1
- list 'use_member' 'wan2_m1_w2'+       option dest_port '443
 +       option proto 'tcp' 
 +        option use_policy 'wan2_wan'
-config 'rule' +config rule 'default_rule
- option 'dest_ip' '0.0.0.0/0' +       option dest_ip '0.0.0.0/0' 
- option 'use_policy' 'wan1_wan2_loadbalanced'+       option use_policy 'balanced'
</code> </code>
Line 472: Line 507:
===== Start mwan3 ===== ===== Start mwan3 =====
-  * mwan3 automatically will start after each reboot but if a reboot has not occurred yet, the package can be manually started.+  * Mwan3 automatically will start after each reboot but if a reboot has not occurred yet, the package can be manually started.
  * see the "Administration" section below   * see the "Administration" section below
Line 480: Line 515:
===== Verification of basic operation ===== ===== Verification of basic operation =====
 +
 +==== Check MWAN3 status in cli ====
 +
 +<code>
 +root@OpenWrt:~# mwan3 status
 +Interface status:
 +Interface wan is online (tracking active)
 +Interface wan2 is online (tracking active)
 +
 +Policy balanced:
 + wan2 (40%)
 + wan (60%)
 +
 +Policy wan1_only:
 + wan (100%)
 +
 +Policy wan2_only:
 + wan2 (100%)
 +
 +Policy wan2_wan:
 + wan2 (100%)
 +
 +Policy wan_wan2:
 + wan (100%)
 +
 +Local connected networks:
 +destination        policy            hits   
 +------------------------------------------------
 +127.0.0.0/8        default            22     
 +224.0.0.0/3        default            0       
 +192.168.1.0/24    default            0       
 +192.168.33.0/24    default            0       
 +213.154.232.8/29  default            0       
 +
 +Active rules:
 +source            destination        proto  src-port      dest-port    policy          hits   
 +---------------------------------------------------------------------------------------------------
 +0.0.0.0/0          213.136.223.128/25 tcp    0:65535      80            wan_wan2        0       
 +1.2.3.4            5.6.7.8            udp    12345:54321  12345:54321  wan2_wan        0       
 +0.0.0.0/0          0.0.0.0/0          all                                balanced        2862   
 +</code>
==== Check status in the MWAN3 overview page ==== ==== Check status in the MWAN3 overview page ====
Line 492: Line 568:
==== Check kernel routing tables ==== ==== Check kernel routing tables ====
-  * "ip route show table 0" should show route tables with table numbers 1000 or higher (e.g. 1018, 1020) -- these tables are generated by mwan3 +  * "ip route show table x" (where x is interface ID) should show a routing table specifically for that interface -- these tables are generated by mwan3
-  * a specific table number can be viewed as desired+
===== Verification of WAN interface load-balancing ===== ===== Verification of WAN interface load-balancing =====

Back to top

doc/howto/mwan3.1383401602.txt.bz2 · Last modified: 2013/11/02 15:13 by timmillerdyck