A Django site.
July 2, 2008
» Automatically Find Fastest Repository Server in Ubuntu 8.04

I’ve blogged about this in the past but it seemed to be overlooked.  I blame the poor choice of title.  In any event, I figure it would be worth revisiting as I’ve had a few suggestions on it being a good topic.

note: Your mileage may vary.  I’ve seen these steps suggest a much faster repository and in other cases they can end up being slower.  Perhaps try two or three until you find a good one.

Updating Your Repository Server

Software packages and security updates are made available in Ubuntu by way of a software repository.  These repositories are mirrored all over the world, many times very close to where you live.  I, for example, have three mirrors within 50 miles of my house.  These are generally much faster than the core Ubuntu repositories.

To find a repository near you, or have your system scan for the fastest (which may or may not be local to you), you will need to launch the Software Sources tool.  This can be found in:

System > Administration > Software Sources

software sources

This administrative menu will allow you to change the location of your repository, and which items within the repository you want to subscribe to.  You will have the most software available if all boxes are checked.

other software sources

From here we’ll want to select “Other” from the drop-down menu and we’ll be provided with a list of available servers.

To have your system try and find the fastest server you can select the “Select Best Server” option, which will try to connect to each repository mirror and track the fastest responses.  The location selected when this process is done should be the fastest available mirror at that time.

select best server

Remember, repository mirror speeds can fluctuate based on traffic and other factors.  For best results you might try this scan periodically and find which mirrors are consistently the best for your location.  Enjoy!

Other Points of Interest

June 10, 2008
» How To Create An Ubuntu Repository Mirror on Ubuntu 8.04

I was discussing the creation and maintenance of an Ubuntu repository mirror in IRC today and realized that I haven’t blogged about how to do that.  Its funny sometimes how topics pop up, and something I’ve been using for months never made it onto the blog.  In any event, below I’ll outline how to create your own Ubuntu repository mirror–great for consolidating and saving bandwidth for multiple machines!

Creating the Mirror

To create an Ubuntu repository mirror we’ll use the apt-mirror tool.  This can be installed by issuing the following command or clicking the link:

sudo aptitude install apt-mirror

Once this package is installed you’ll need to configure a few items:

  1. Where will your mirror be stored?
  2. How aggressive will you be about downloading?
  3. What do you want to mirror?

To edit these values you’ll want to edit the /etc/apt/mirror.list file.  I’ve displayed mine below, and broken it down into the three main sections.

Where Will Your Mirror Be Stored?

I’ve configured my apt-mirror to be stored on an external USB drive.  This is defined via the line:

set base_path      /media/gurudisk/UBUNTU/MIRROR

This overrides the default value, which stores the data in /var/spool/apt.

How Aggressive Will You Be About Downloading?

This title may be a bit misleading but this is the way I think about it.  Technically its the number of download threads you want to use.  The higher the number, the more threads, the faster its done.  This will depend on your bandwidth connection though.  On my home DSL I can’t put this number any higher than 2.  In faster locations you’ll be able to put the number much higher.  Your milage may vary.

set nthreads     2

What Do You Want To Mirror?

Now, the most important part, is what do you actually want to mirror?  The syntax for this section is pretty simple.  It looks just like a normal sources.list entry, but for each address you want to mirror.  Here is an example of my entry for Ubuntu 8.04, i386:

# Ubuntu 8.04 "Hardy" i386
deb-i386 http://archive.ubuntu.com/ubuntu hardy main restricted universe multiverse
deb-i386 http://archive.ubuntu.com/ubuntu hardy-updates main restricted universe multiverse
deb-i386 http://archive.ubuntu.com/ubuntu hardy-backports main restricted universe multiverse
deb-i386 http://archive.ubuntu.com/ubuntu hardy-security main restricted universe multiverse
deb-i386 http://archive.ubuntu.com/ubuntu hardy-proposed main restricted universe multiverse

If you want amd64 bit mirrored you might add entries like this:

# Ubuntu 8.04 "Hardy" amd64
deb-amd64 http://archive.ubuntu.com/ubuntu hardy main restricted universe multiverse
deb-amd64 http://archive.ubuntu.com/ubuntu hardy-updates main restricted universe multiverse
deb-amd64 http://archive.ubuntu.com/ubuntu hardy-backports main restricted universe multiverse
deb-amd64 http://archive.ubuntu.com/ubuntu hardy-security main restricted universe multiverse
deb-amd64 http://archive.ubuntu.com/ubuntu hardy-proposed main restricted universe multiverse

Notice that I’m mirroring main, restricted, universe and multiverse plus the security, proposed, backports and updates sections.  This gives me a complete mirror of everything that would normally be available.  If you don’t want/need proposed or backports, etc you can simply remove those lines.

If you want to be able to use your mirror to support network based installations you’ll also need to add the following (depending on your architecture):

deb-amd64 http://archive.ubuntu.com/ubuntu hardy main main/debian-installer
deb-amd64 http://archive.ubuntu.com/ubuntu hardy restricted restricted/debian-installer
deb-amd64 http://archive.ubuntu.com/ubuntu hardy universe universe/debian-installer
deb-amd64 http://archive.ubuntu.com/ubuntu hardy multiverse multiverse/debian-installer

or

deb-i386 http://archive.ubuntu.com/ubuntu hardy main main/debian-installer
deb-i386 http://archive.ubuntu.com/ubuntu hardy restricted restricted/debian-installer
deb-i386 http://archive.ubuntu.com/ubuntu hardy universe universe/debian-installer
deb-i386 http://archive.ubuntu.com/ubuntu hardy multiverse multiverse/debian-installer

Basically just add the sources.list entry to this file for anything you want to mirror.  I also mirror Medibuntu and a few PPA locations.  It sure makes for faster installation when I am able to use the LAN as compared to the public tubes.

To keep your mirror updated activate the cron entry in /etc/cron.d/apt-mirror.  By default it will try to update your mirror at 4:00am, and it is disabled.  To activate it simply uncomment the line and change the 4 to the preferred hour of your choice.  Enjoy!

Update: My mirror, including main, restricted, universe, multiverse, -proposed, -backports, -security, Medibuntu and the network installation sections takes up 37G.  I do mirror i386 and amd64, so if you just need one architecture you could half-that.

Random Posts

October 17, 2007

Dennis Muhlestein
nonic
All My Brain
» Using piped svndumpfilter commands to separate an svn repository

According to the documentation for svndumpfilter, you can include one subcommand when filtering a dumped repository. Suppose you have a repository that has a path “/some/path” that you’d like to separate out into its own new repository. From the documentation, you simply pipe the original dumped repository through the svndumpfilter command. Example: cat [...]