User Tools

Site Tools


doc:howto:proxy.squid
This wiki is read only and for archival purposes only. >>>>>>>>>> Please use the new OpenWrt wiki at https://openwrt.org/ <<<<<<<<<<

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
doc:howto:proxy.squid [2017/05/18 12:16]
tmomas Section heading levels corrected (there shall be one and only one level 1 heading)
doc:howto:proxy.squid [2018/02/09 12:55] (current)
Jalakas [Configuration]
Line 2: Line 2:
 Squid is an enterprise-class caching web proxy.  ​ Squid is an enterprise-class caching web proxy.  ​
  
-===== 1. Squid transparent mode on devices with sufficient ​flash space to install Squid ===== +===== 1. Squid transparent mode on devices with sufficient space to install Squid ===== 
-==== Install Squid on LEDE/​OpenWrt device ​====+==== Prerequisites ==== 
 +=== External storage === 
 +You will need additional storage for Squid cache. 
 + 
 +==== Installation ==== 
 +Install Squid on LEDE/​OpenWrt device ​(can be extrooted for more installation space):
 <​code>​ <​code>​
 opkg install squid opkg install squid
 </​code>​ </​code>​
  
-=== Caching device === +Optional packages: 
-Add cache device (/dev/sda1 in this example).+  * luci-app-squid - Luci application for managing Squid settings 
 +  * squid-mod-cachemgr - Page for Squid statistics etc 
 +<​code>​ 
 +opkg install luci-app-squid squid-mod-cachemgr 
 +</code>
  
-Mount it to "/​tmp/​squid":​+==== Storage configuration ==== 
 +You need to update '​fstab'​ configuration to mount your caching storage device partition ​to "/​tmp/​squid"​
 + 
 +Open '​fstab'​ configuration in Luci or in terminal:
 <​code>​ <​code>​
 vi /​etc/​config/​fstab vi /​etc/​config/​fstab
 </​code>​ </​code>​
  
 +In this example partition '/​dev/​sda1'​ with '​ext4'​ filesystem, is mounted to '/​tmp/​squid',​ and filesystem check (fsck) is enabled:
 <​code>​ <​code>​
 config mount config mount
Line 22: Line 35:
         option fstype '​ext4'​         option fstype '​ext4'​
         option enabled_fsck '​1'​         option enabled_fsck '​1'​
-        option target '/​tmp/​squid+        option target '/​tmp/​squid'
 </​code>​ </​code>​
  
-=== Add forwarding ​to Squid into firewall ​=== +Save your configuration and try out if mounting works. Manually mount your configuration file for test: 
-Edit firewall section: ​+<​code>​ 
 +mount -a 
 +</​code>​ 
 + 
 +And check if you see your device in list: 
 +<​code>​ 
 +df -h 
 +</​code>​ 
 +=== Set up forwarding === 
 +Add http (port 80) traffic forwarding to Squid (so called transparent mode).  
 +Add firewall section: ​
 <​code>​ <​code>​
 vi /​etc/​config/​firewall vi /​etc/​config/​firewall
Line 49: Line 72:
 </​code>​ </​code>​
  
-=== Configuration ​===+=== Squid configuration ​===
 Edit Squid configuration:​ Edit Squid configuration:​
 <​code>​ <​code>​
 vi /​etc/​squid/​squid.conf vi /​etc/​squid/​squid.conf
 </​code>​ </​code>​
-or install ​**luci-app-squid** and go to Services->​Squid->​Advanced Settings.+or use **luci-app-squid** and go to Services->​Squid->​Advanced Settings.
  
 <​code>​ <​code>​
Line 89: Line 112:
  
 http_access deny all http_access deny all
- 
-# Allow ICP queries from local networks only 
-icp_access allow localnet 
-icp_access deny all 
  
 refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^ftp: 1440 20% 10080
Line 121: Line 140:
 You can use your device capacity here, but make sure you leave some (5% - 15%) free space on device for logs/folder tree etc.  You can use your device capacity here, but make sure you leave some (5% - 15%) free space on device for logs/folder tree etc. 
  
-:!: Squid will crash when the disk gets full or unwritable, all your Internet traffic, also access to Luci will stop working. You can fix it by logging in with SSH, disabling cache_dir: <​code>#​cache_dir aufs /​tmp/​squid/​cache 900 16 512</​code>​ or fixing settings. +:!: Squid will crash when the disk gets full or unwritable!  
-Reload ​is needed after changes or disabling.+When this happens ​your Internet traffic, also access to Luci will stop working. You can fix it by logging in with SSH, disabling cache_dir: ​ 
 +<​code>#​cache_dir aufs /​tmp/​squid/​cache 900 16 512</​code>​ or by fixing settings. 
 +Configuration reload ​is needed after all changes or disabling.
  
-==== Reload configuration ​and rebuild cache ===== +=== Reload configuration ==== 
-Reload ​Squid configuration:+Squid reconfiguration and cache directory tree rebuild ​:
 <​code>​ <​code>​
-squid -k reconfigure ​   (use -f cfgfile if it has moved) ​    +squid -k reconfigure ​   (use -f "cfgfile" ​if congiguration file has moved) ​    
-squid -z                (rebuild cache directory) +squid -z                (rebuild cache directory ​tree
-squid                   ​(start squid to make sure it'​s ​running)+squid                   ​(start squid to make sure it will be running)
 </​code>​ </​code>​
 +
 +=== Keep settings ===
 +If you have successfully set up your Squid cache, you may want to preserve the settings while (future) sysupgrade.
 +Add squid configuration **/​etc/​squid/​squid.conf** into sysupgrade keep file: 
 +<​code>​
 +vi /​etc/​sysupgrade.conf
 +</​code>​
 +or use **luci** and go to System->​Backup/​Flash Firmware->​Configuration.
 +
 +<​code>​
 +## This file contains files and directories that should
 +## be preserved during an upgrade.
 +
 +# /​etc/​example.conf
 +# /​etc/​openvpn/​
 +
 +/​etc/​squid/​squid.conf
 +</​code>​
 +
 +==== Execution ====
 +Squid should be working now.
 +If you want to be sure it's really caching, you can control cache folder used size:
 +<​code>​
 +df -h
 +</​code>​
 +
 +That should be something like in this example:
 +<code bash>
 +Filesystem ​               Size      Used Available Use% Mounted on
 +        ..                  ..        ..        ..   ​.. ​        ..
 +/​dev/​sda1 ​              ​1000M ​      ​600M ​     400M  60% /tmp/squid
 +</​code>​
 +
 +Or use squid-mod-cachemgr,​ accessing its page: 
 +<​code>​
 +http://​192.168.1.1/​cgi-bin/​cachemgr.cgi
 +</​code>​
 +
  
 ===== 2. Squid on devices without enough flash space ===== ===== 2. Squid on devices without enough flash space =====
 :!: This howto is a work in progress, and I expect that it will not work for everyone unless others (i.e. you) contribute to it's development. :!: This howto is a work in progress, and I expect that it will not work for everyone unless others (i.e. you) contribute to it's development.
  
-==== Prerequisities ​====+==== Prerequisites ​====
 === External storage === === External storage ===
 You //will// need additional storage for Squid, definitely for it's cache, and most likely for the executable too.  ​ You //will// need additional storage for Squid, definitely for it's cache, and most likely for the executable too.  ​
  
-This howto assumes that an ext4 filesystem is mounted as **/opt**, with at least 4GB of free storage space. ​ IMHO, this is //much// easier than using [[doc:​howto:​extroot:​|extroot]].+This howto assumes that an ext4 filesystem is mounted as **/opt**, with at least 4GB of free storage space. IMHO, this is //much// easier than using [[doc:​howto:​extroot:​|extroot]].
  
 <code bash> <code bash>
Line 158: Line 217:
  
 === Before installing === === Before installing ===
-For maximum compatibility,​ install Squid'​s dependencies in the regular way so that Squid and all other apps will find them in the expected location. ​ Depending upon which packages you have already installed, these dependencies may already be on the rootfs.  ​+For maximum compatibility,​ install Squid'​s dependencies in the regular way so that Squid and all other apps will find them in the expected location. Depending upon which packages you have already installed, these dependencies may already be on the rootfs.  ​
  
-To do this, execute the following command:<​code ​ bash>+To do this, execute the following command: 
 +<code bash>
 root@db-router:​~#​ opkg install $(opkg depends -A squid* | grep -v depends | grep -v squid | sort -u) root@db-router:​~#​ opkg install $(opkg depends -A squid* | grep -v depends | grep -v squid | sort -u)
 Package libc (1.1.11-1) installed in root is up to date. Package libc (1.1.11-1) installed in root is up to date.
Line 170: Line 230:
 </​code>​ </​code>​
  
-You will then need to ensure packages can be installed onto the external storage:<​code ​ bash>+You will then need to ensure packages can be installed onto the external storage: 
 +<code bash>
 ### Allow packages to be installed to (external storage mounted as) /opt... ### Allow packages to be installed to (external storage mounted as) /opt...
   if ! grep -q usb /​etc/​opkg.conf;​ then   if ! grep -q usb /​etc/​opkg.conf;​ then
Line 182: Line 243:
  
 === Installing === === Installing ===
-You can install the Squid packages via the following command:<​code bash>+You can install the Squid packages via the following command: 
 +<code bash>
 ### Install the Squid package (and optionally, the cache manager package)... ### Install the Squid package (and optionally, the cache manager package)...
   opkg -d usb install squid   opkg -d usb install squid
Line 194: Line 256:
 Because the Squid package is installed on external storage (e.g. the executable is in **/​opt/​usr/​sbin** instead of **/​usr/​sbin**),​ we need to do a few tricks for it to work. Because the Squid package is installed on external storage (e.g. the executable is in **/​opt/​usr/​sbin** instead of **/​usr/​sbin**),​ we need to do a few tricks for it to work.
  
-There are several ways to achieve this (such as adding **/​opt/​usr/​sbin** to **$PATH**), but I recommend:<​code bash>+There are several ways to achieve this (such as adding **/​opt/​usr/​sbin** to **$PATH**), but I recommend: 
 +<code bash>
 ### Create a link to the startup script and configuration files... ### Create a link to the startup script and configuration files...
   ln -s /​opt/​etc/​init.d/​squid ​     /​etc/​init.d/​squid ​       ## this is absolutely required (i.e. dont cp)   ln -s /​opt/​etc/​init.d/​squid ​     /​etc/​init.d/​squid ​       ## this is absolutely required (i.e. dont cp)
Line 225: Line 288:
 </​code>​ </​code>​
  
-Squid should now run (well, not yet. Keep reading :​-)):<​code bash>+Squid should now run (well, not yet. Keep reading :-)): 
 +<code bash>
 /​etc/​init.d/​squid start /​etc/​init.d/​squid start
 ps -w | grep squid ps -w | grep squid
Line 237: Line 301:
 ==== Configuration ==== ==== Configuration ====
  
-For reference, OpenWrt'​s default Squid configuration can always be found on GitHub, including the [[https://​github.com/​openwrt/​packages/​blob/​master/​net/​squid/​files/​squid.init|init script]], the [[https://​github.com/​openwrt/​packages/​blob/​master/​net/​squid/​files/​squid.config|uci configuration]],​ and [[https://​github.com/​openwrt/​packages/​blob/​master/​net/​squid/​files/​squid.conf|squid.conf]]. ​ :!: Squid will //not run// from external storage unless some changes are made to these files (some of these changes have been made in earlier sections).+For reference, OpenWrt'​s default Squid configuration can always be found on GitHub, including the [[https://​github.com/​openwrt/​packages/​blob/​master/​net/​squid/​files/​squid.init|init script]], the [[https://​github.com/​openwrt/​packages/​blob/​master/​net/​squid/​files/​squid.config|uci configuration]],​ and [[https://​github.com/​openwrt/​packages/​blob/​master/​net/​squid/​files/​squid.conf|squid.conf]]. 
 +:!: Squid will //not run// from external storage unless some changes are made to these files (some of these changes have been made in earlier sections).
  
 === Creating the cache directory === === Creating the cache directory ===
Line 270: Line 335:
 This may be useful for debugging configurations,​ etc. This may be useful for debugging configurations,​ etc.
  
-You might not get the Squid executable to work from the command line (e.g. **squid -d2**) without (temporarily) adding the following to **/​etc/​squid/​squid.conf**:<​code>​+You might not get the Squid executable to work from the command line (e.g. **squid -d2**) without (temporarily) adding the following to **/​etc/​squid/​squid.conf**:​ 
 +<​code>​
 http_port 3128 http_port 3128
 coredump_dir /tmp/squid coredump_dir /tmp/squid
Line 277: Line 343:
 </​code>​ </​code>​
  
-A better option may be to start/stop Squid, and then use the generated configuration:<​code bash>+A better option may be to start/stop Squid, and then use the generated configuration:​ 
 +<code bash>
 /​etc/​init.d/​squid start /​etc/​init.d/​squid start
 /​etc/​init.d/​squid stop /​etc/​init.d/​squid stop
Line 288: Line 355:
  
 ==== Before a sysupgrade ==== ==== Before a sysupgrade ====
-Because Squid (and it's config file) is on external (i.e. permanent) storage, I suggest you **do not** do something similar to the following:<​code bash>+Because Squid (and it's config file) is on external (i.e. permanent) storage, I suggest you **do not** do something similar to the following: 
 +<code bash>
 # echo '/​etc/​squid/'​ > /​lib/​upgrade/​keep.d/​squid ​                  ## Keep config across sysupgrades # echo '/​etc/​squid/'​ > /​lib/​upgrade/​keep.d/​squid ​                  ## Keep config across sysupgrades
 </​code>​ </​code>​
Line 295: Line 363:
  
 ==== After a sysupgrade ==== ==== After a sysupgrade ====
-After a **sysupgrade**,​ the following may need doing before Squid will run (NB: this assumes you're using **/​opt**):<​code bash>+After a **sysupgrade**,​ the following may need doing before Squid will run (NB: this assumes you're using **/opt**): 
 +<code bash>
 # the following needs redoing after a sysupgrade # the following needs redoing after a sysupgrade
 [ -h /etc/squid ]                || rm -rf /​etc/​squid ​                            > /dev/null 2>&1 ## if not a symlink, delete the dir  [ -h /etc/squid ]                || rm -rf /​etc/​squid ​                            > /dev/null 2>&1 ## if not a symlink, delete the dir 
Line 321: Line 390:
 The following requires that **uhttpd** is installed and running (the simplest/​safest way to do this is **opkg install luci**). The following requires that **uhttpd** is installed and running (the simplest/​safest way to do this is **opkg install luci**).
  
-Do the following:<​code bash>+Do the following: 
 +<code bash>
 opkg -d usb install squid-mod-cachemgr opkg -d usb install squid-mod-cachemgr
 echo 127.0.0.1:​3128 > /​etc/​squid/​cachemgr.conf echo 127.0.0.1:​3128 > /​etc/​squid/​cachemgr.conf
Line 332: Line 402:
 This is how to use a user:group other that nobody:​nogroup (**FYI only, not recommended**). This is how to use a user:group other that nobody:​nogroup (**FYI only, not recommended**).
  
-First, create the user and group:<​code bash>+First, create the user and group: 
 +<code bash>
 # Create Squid user & group... # Create Squid user & group...
   opkg install shadow --force-overwrite # provides groupadd & useradd utils, but must replace passwd   opkg install shadow --force-overwrite # provides groupadd & useradd utils, but must replace passwd
Line 353: Line 424:
 </​code>​ </​code>​
  
-Then you need to configure external storage (permissions,​ and owner:​group):<​code bash>+Then you need to configure external storage (permissions,​ and owner:​group):​ 
 +<code bash>
 ### create cache directory (may already exist) ### create cache directory (may already exist)
   mkdir -p          /​opt/​var/​cache/​squid   mkdir -p          /​opt/​var/​cache/​squid
Line 360: Line 432:
 </​code>​ </​code>​
  
-The you need to configure squid to use the new user:​group:<​code bash>+The you need to configure squid to use the new user:group: 
 +<code bash>
 ### Make the necessary changes to the configuration file (redoable)... ### Make the necessary changes to the configuration file (redoable)...
 if ! grep -q cache_effective_user /​etc/​squid/​squid.conf;​ then if ! grep -q cache_effective_user /​etc/​squid/​squid.conf;​ then
doc/howto/proxy.squid.1495102592.txt.bz2 · Last modified: 2017/05/18 12:16 by tmomas