Debian/Ubuntu using apt proxy to save bandwidth

If you have a home network or a small office network like me, you may be have more than just one Linux machine.

In my case, I have 9 embedded machines (Raspberry PI, Cubieboard, Cubietruck, ODROID-U3) as well as 2 Linux Desktop’s, 1 Linux Notebook and one server which is running several virtual environments on demand.

As you can see, these are quite many machines, which are of course not running 24×7. However they still need updates here and there and specially the newly created virtual machines are always require to download a big chunk of updates after they are installed.

If you then live in a rural environment like me, the internet bandwidth is also very limited.

To get around this a proper solution is to use a proxy server. I really hated using proxy servers for web-browsing, but in case of getting and storing deb packages centralized it is really perfect!

Solution and Hardware:

The simple solution is Apt-Cacher-NG. You just need to find a central server which will run all the time and does have a bit of space to store the deb-packages.
In my case, I used one of my Cubietrucks, which has a 128 Gb SSD Drive attached. This little server consumes below 3 watts including the SSD drive, so it doesn’t harm when it runs 24×7.

Installation:

After installing apt-cacher-ng, the default configuration in Ubuntu as well as in Debian is:
/etc/apt-cacher-ng/  <- Configuration
/var/log/apt-cacher-ng/ <- Log files
/var/run/apt-cacher-ng/ <- Pid-file, Socket file
/var/cache/apt-cacher-ng/ <- .deb file repository

There is also an additional UNIX user “apt-cacher-ng” added which is not allowed to login. You can see that user at /etc/passwd.

Minimal customization

As you can see, right after installing apt-cacher-ng it is already up and running, but it may be not perfectly set for your server.

If you have a look at above directories the only space-intensive one is /var/cache/apt-cacher-ng .

My Cubietruck is limited to 32gb on all directories, except /home, which is the mounted 128 GB SSD. So I want to point /var/cache/apt-cacher-ng to /home/apt-cacher-ng.

So first of all stop the already running service:

sudo /etc/init.d/apt-cacher-ng stop

There are obviously two ways to do it:

a) move away /var/cache/apt-cacher-ng to /home and and change the entries in the config file /etc/apt-cacher-ng/acng.conf and the homedir setting in /etc/passwd

sudo mv /var/cache/apt-cacher-ng /home
sudo vi /etc/apt-cacher-ng/acng.conf
sudo vi /etc/passwd

<or>

b) move away /var/cache/apt-cacher-ng to /home and create a symlink back to the old directory.

sudo mv /var/cache/apt-cacher-ng /home
sudo ln -s /home/apt-cacher-ng /var/cache

I chosen the symbolic link way as it will keep all automatic changes to the config files due to system-upgrades easier. It is indeed the lazy guy solution, but is fine for my home environment.

Then just start up the service again:

sudo /etc/init.d/apt-cacher-ng start

Voila! Your apt-cacher-ng is running again and we can continue.

Client Installations

The first *best* client is of course the server itself. What you need is to let all “apt” tools (Aptitude, apt-get, apt-cache, Synaptics and so on) know that they should use a proxy server.
This will be done by a simple configuration change in /etc/apt/apt.conf.d/ . You just need to create a new file which includes the additional configuration.
In my case it is “02proxy”  and includes just following line:

Acquire::http { Proxy "http://littleserver.mynet.loc:3142"; };

However, this little change doesn’t work for you directly, as there must be the correct server entered.

So figure out how your server is reachable by all your machines. If that server doesn’t have a DNS name, it may be using the same IP-address all the time. One of both ways is required.

In my case, the ip address is indeed “dynamic” as all my machines are using dhcp, but I set my local router to always give this machine the same ip address.
So my cubietruck server does have following IP-address:
192.168.0.2

But since I am also running a dns service on my router, I can also reach my cubietruck server from each computer in my local network with it’s DNS name, which I can check with:

daniel@littleserver:/home/daniel$ hostname -A 
littleserver.mynet.loc

So if in your case you only have the ip address the configuration file /etc/apt/apt.conf.d/02proxy may look like:

Acquire::http { Proxy "http://192.168.0.2:3142"; };

That’s it! now every new package will be cached via the proxy.

Testing the Proxy:

The easiest way is of course installing a new package.

To ensure that the proxy is running, reload the package list:

apt-get update

It should look just as usual, but if you see something like this:

Unable to connect to littleserver.mynet.loc:3142:

Then maybe:
* Your apt-cacher-ng is not running at all
* The client configuration above does not fit to your server
* Your Server is not reachable

If everything looks fine, you can just install something ( i.e. lynx, w3m, wget or speex).
When you see a “real” download and installation, you can look out for the same file in your cache directory.

If that works fine, just get over to all your linux machines and add the same /etc/apt/apt.conf.d/02proxy file as mentioned above in the client installation section.

Thats it! There is nothing more to do.
🙂

Frequently Asked Questions:

1) Can I use Debian and Ubuntu clients pointing to the same proxy?

Yes, no problem with that. All files are stored in a separate directory, they won’t interfere each other. So also machines running older and newer Ubuntu/Debian versions can share the same proxy.

2) My Clients are i386, x64 and Arm architecture. Do I get problems downloading wrong files ?

No, as said in Question 1, the files are stored separately and it doesn’t matter how you mix it up.

3) Does it make sense to harmonize my apt-sources lists ?

Yes, it make sense to ensure that similar machines use the same repositories, so that they will point to the same servers and utilize already downloaded files.

 

More questions ? Just let me know 🙂

 

Leave a Reply

Your email address will not be published. Required fields are marked *

Protected with IP Blacklist CloudIP Blacklist Cloud