Lyte's Blog

Bad code, bad humour and bad hair.

Dropping Repoproxy Development for Apt-cacher-ng

Initially I started writing repoproxy because I thought there were a bunch a of features I needed that simply couldn’t be configured with apt-cacher-ng (ACNG), it turns out I was wrong.

The reasons I had (broadly) for starting repoproxy:

- shared caches - multiple caches on a LAN, they should share some how

- CentOS - CentOS uses yum, how could it possibly be supported by ACNG?

- roaming clients - e.g. laptops that can't always sit behind the same static instance of ACNG

- NodeJS - I wanted to learn NodeJS, I've learnt a bit now and don't feel a huge desire to complete this project given I've figured out most of the other features neatly

Below I cover how to achieve all of the above features with ACNG as it really wasn’t obvious to me so I suspect it might be useful to other too.

Shared Caches

I run a lot of VMs on my laptop, but also have other clients in the home and work networks that could benefit from having a shared repository cache. For me a good balance is having static ACNG servers at both home and work, but also having ACNG deployed to my laptop so that the the VMs can be updated, reimaged or get new packages without chewing through my mobile bandwidth.

This is actually natively supported with ACNG, it’s just a matter of putting a proxy line in /etc/apt-cacher-ng/acng.conf like so:

1
Proxy: http://mirror.lyte:3142/

Then it’s just a matter of telling apt to use a localhost proxy anywhere that ACNG is installed:

1
echo 'Acquire::http { Proxy 'http://localhost:3142/'; };' > /etc/apt/apt.conf.d/01proxy

This allows VMs on my laptop to have a portable repository cache when I’m not on a normal network, but also allows me to benefit from cache others generate and vice versa.

I’d like to at some point have trusted roaming clients (i.e. only my laptop) publish captured ACNG cache back to static ACNG cache servers. I’m pretty sure I can achieve this using some if-up.d trickery combined with rsync, but I haven’t tried yet.

I had considered trying to add something more generic to repoproxy that did a cache discovery and then possibly ICP (Internet Cache Protocol) to share cache objects between nodes on the same LAN, but there are some generalised security issues I can’t come up with a good solution for, e.g. If my laptop is connected to a VPN and another node discovers it, how do I sensibly ensure they can’t get anything via the VPN without making the configuration overly obtuse?

It seems like trying to implement self discovery would either involve a lot of excess configuration on participating nodes, or leave gaping security holes so for the moment I’m keeping it simple.

CentOS

I use CentOS and Scientific Linux sometimes and I’d like their repos to be cached too.

I had originally falsely assumed this would simply be impossible, but I read somewhere that there was at least a little support.

In my testing some things didn’t work out of the box, but could be worked around.

Essentially it seems like ACNG treats most files as one of volatile, persistent or force-cached and it’s just a matter of relying on tweaking URL based regexs to understand any repositories you want to work with.

Roaming Clients

When I move my laptop between home and work or on to a public wifi point I want apt to “just work” I don’t want to have to remember to alter some config each time I need it.

I found two methods described on help.ubuntu.com that were sort of on the right track, but running a cron every minute that insights network traffic when my network is mostly stable seems like a bad idea, as does having to reboot to gain the new settings (especially as Network Manager won’t bring wireless up until after I login, so it wouldn’t even work for my use case).

NB: I’ve already gone back and added my event driven method to the help.ubuntu.com article.

I suspect it would be possible to utilise ACNG’s ´╗┐´╗┐hook functionality:

8.9 How to execute commands before and after going online? It is possible to configure custom commands which are executed before the internet connection attempt and after a certain period after closing the connection. The commands are bound to a remapping configuration and the config file is named after the name of that remapping config, like debrep.hooks for Remap-debrep. See section 4.3.2, conf/.hooks and /usr/share/doc/apt-cacher-ng/examples/.hooks files for details.

I couldn’t immediately bend this to my will, so I decided to go down a route I already understood.

I decided to use if-up.d to reset ACNG’s config every time there was a new interface brought online, this allows for an event-driven update of the upstream proxy rather than relying on polling intermittently or a reboot of the laptop.

Create a new file /etc/network/if-up.d/apt-cacher-ng-reset-proxy and put the following script in it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
#!/bin/bash

# list of hosts that the proxy might be running on
hosts=(
        acng.on.my.home.network
        acng.on.my.work.network
        acng.at.my.friends.place
)

set_host() {
        host=$1
        line='Proxy: http://$host:3142/'
        if [[ -z $host ]]; then
                line='# Proxy: disabled because none are contactable'
        fi

        # adjust ACNG configuration to use supplied proxy
        sed -i -r 's%^\s*(#|)\s*Proxy: .*$%$line%g' \
                /etc/apt-cacher-ng/acng.conf

        # if apt-cacher-ng is running
        if service apt-cacher-ng status > /dev/null 2>&1; then
                # restart it to take hold of new config
                service apt-cacher-ng restart
        fi
        exit 0
}

try_host() {
        host=$1
        # if we can get to the supplied host
        if ping -c 1 '$host' > /dev/null 2>&1; then
                # tell ACNG to use it
                set_host '$host'
        fi
}

# Run through all possible ACNG hosts trying them one at a time
for host in '${hosts[@]}'; do
        try_host '$host'
done

# no proxies found, unset upstream proxy (i.e. we connect straight to the internet)
set_host

Make sure to adjust the script for your environment and make it executable with:

1
chmod +x /etc/network/if-up.d/apt-cacher-ng-reset-proxy

Comments