Howto: Download packages and dependencies for offline installation

From my perspective, offline installation is the weakest spot in the Linux superstructure. I understand the shared libraries concept, and I think it’s a marvelous thing, but it doesn’t erase the fact that if you’re not in a position for at least intermittent Internet access, it’s a dealbreaker.

Dependencies are almost impossible to track, and are usually scattered. The more complex a program is, the more dependencies it will have. Furthermore, those dependencies will probably have dependencies, and those have to be macromanaged too, if you can’t rely on aptitude to handle it for you.

There’s an added problem when you realize that an offline machine can’t be sure what versions are new. Until aptitude synchronizes with the repositories, the packages in its database are possibly obsolete.

In this solution, the idea is to create a web page that links to the Ubuntu Package Search pages, which can be opened on a machine that’s connected, and download the software plus its dependencies without too much hassle. From there, the packages can be ferried back to the offline box and installed manually with dpkg.

The first stumbling block to the concept is that there is no way to hard link straight to a deb package, since the offline machine can’t build links straight to it. The version number is unpredictable, and changes frequently within each release. Even more perplexing is the indexing system used on the repositories, where software is split alphabetically into subdirectories. If a package were to move, or the software to change its name, it’s altogether possible that the package might shift to a new location. That, again, is something an offline machine can’t foresee.

However, there is a mechanism on the search pages that can help. There is a built-in search option for software, accessible by tacking the package name on to the end of the URL. For example, to search for the leafpad package, you only need enter “http://packages.ubuntu.com/leafpad” into the address bar. The search mechanism with parse that automatically, and give you links to the software page, for each Ubuntu release.

So this method queries apt-cache on an offline machine with the depends option. The output is redirected back into apt-cache depends, then again for a third level of dependencies. Hopefully this will catch any oddball packages you might need.

apt-cache’s output is rearranged into a workable list with sort, then filtered with a short (and probably clumsy) mawk program that cuts out the gunk and leaves you with a sparse html document that links to the package search pages.

From there, you’re within three clicks of the actual deb file, and once you have all the dependencies downloaded, you can transfer them back to your offline machine. Invoke them with dpkg -i *, and barring incident, it should install the software you asked for, without too much difficulty.

This method is not foolproof. I’ve tested it twice now, once with leafpad and its dependencies, and once with Critical Mass. Both times it worked fine. I also tested it with xorg, although I admit I didn’t ferry all of xorg’s dependencies back to my offline machine. There were just too many, and by then I was fairly confident it was working.

Anyway, it’s altogether possible that you’ll get a version mismatch, or a dependency that’s even deeper than three levels, that you didn’t download on the first try. I’m sorry. Just rerun the script using that package as the target, and after installing that software and its dependencies, your first package should work fine. Lather, rinse, repeat.

This isn’t much use installing software that doesn’t show up in apt’s repositories to start with. In other words, you can’t install Critical Mass out of universe if you haven’t synced your online machine with universe. Apt won’t know what universe is, doesn’t know Critical Mass is in it — for that matter, doesn’t even know Critical Mass exists. (The workaround to this is to download the metapackage for a game or an application, and dpkg it on the offline machine. That should tell apt what dependencies it needs, and from there you can run the script. … Does that make sense? )

This also won’t help much with upgrades or dist-upgrades, since again, a true offline machine has no idea whatsoever of what’s been improved or rereleased since it was born. If you want to meld your own system for handling upgrades and dist-upgrades, be my guest.

Lastly, I feel I should mention that neither the script and mawk program uses sudo, nor does either program use any software beyond an initial installation. In other words, you should be able to access apt-cache, sort and mawk from any Ubuntu machine … unless you threw out one of those packages. All of this — the script, for what it’s worth, and the program — are GPL 3.0, and you’re welcome to use it in your own Ubuntu adventures, however you see fit.

deps-script.sh Ubuntu pastebin version
PKGNAME=leafpad

apt-cache depends $PKGNAME | xargs apt-cache depends 2>/dev/null | xargs apt-cache depends 1>$PKGNAME-deps.txt 2>/dev/null
sort -d -u -b -o $PKGNAME-deps-sorted.txt $PKGNAME-deps.txt
mawk -f deps2html.mawk $PKGNAME-deps-sorted.txt > $PKGNAME-links.html

deps2html.mawk Ubuntu pastebin version
# deps2html.mawk
# 24 July 2007 by K.Mandla
# Released under GPL 3.0
# See http://www.gnu.org/licenses/gpl-3.0.txt for information and terms of the license.

# This is a very simple mawk program that sifts through the output of
# apt-cache depends, and builds a hyperlinked page that can be useful
# for installing software on offline machines. Of course, it's only
# practical to a point, since there's no way of hotlinking to a .deb
# package (because of the naming convention) and any offline machine
# is probably going to have an outdated package list anyway.

{
# First take out all the spaces, since that interferes with the creation
# of hyperlinks. Four spaces is more than enough, since it's very rare
# that more than four occur together, and if they do, they generally
# are in a line that will be removed anyway.

for (i = 1 ; i = 5 ; i++) gsub(" ","")

# Screen out anything marked as a dependency, since it's lower than
# we're willing to go. Also cut other marked packages, since those are
# either irrelevant or just in the way. The echo command is really only
# a dummy, kicking mawk to the next record without doing anything.
# There are probably better ways to do that.

if (match ($0,"Depends:") || match ($0,"Conflicts:") || match ($0,"Predepends:") || match ($0,"Suggests:") || match ($0,"Replaces:") || match ($0,"Recommends:") || match ($0,"Breaks:")) echo

# If it's a valid package and a reasonable dependency, build the link.
else print "[a href=\"http://packages.ubuntu.com/"$0"\" /]"$0"[/a][br /]"
}

Step by step: First, download or copy-and-paste the script and program into a file. Please note that you’re not required to use the script, although it does offer a quick way to get the entire business done. Otherwise, you’d have to type in some commands, and nobody likes to use the command line. I mean … er … yeah. Anyway.

If you copy and paste the mawk program, change the brackets in the final line to greater-than and less-than symbols. Once again, WordPress’s coding is in the way of my coding. If you download the script and program, cut off the .doc extension and add the dot back to where it ought to be — .sh or .mawk. Once again, WordPress’s ridiculous filename requirements are in the way of my blogging. …

Next, move them to the offline machine. Edit the script to change the PKGNAME variable to the name of your program, then run it with sh deps-script.sh. It will do all the hard work, and you’ll be left with three new files: PKGNAME-deps.txt, which is the raw output from apt-cache; PKGNAME-deps-sorted.txt, which is the output after the sort pass; and PKGNAME-links.html, which is a headerless, bodyless list of dependencies, coded as html links, that form a very simple Web page when opened in a browser.

Move the links file to an online machine — any type, any architecture, any operating system — and open it in a browser. Follow each link, picking your release and architecture, and finally following the download link. Download all your software packages, and ferry them back to the offline machine.

From there, you should be able to install your software manually, with sudo dpkg -i *, or a variation of those commands. This part is where you’re on your own, and if you’re lucky (type with your fingers crossed), your software should drop into place without a hitch. Ta-da! You just installed software on an offline machine, courtesy of apt-cache and mawk.

I hope this works for you. I know the offline-installation issue isn’t a huge deal for most people, now that we’ve entered a golden age of the Internet. Still, it’s always possible that someone, somewhere — like me — is working in an offline environment, and needs a way to get an application installed without access to the ‘Web. If it works, let me know. If it doesn’t, let me know. Cheers and good luck.

P.S.: No making fun of my programming skills. I hit the pinnacle of my programming career when I typed “Hello World” into my C-64. So just cool it. On the other hand, if you have honest suggestions, I’m always willing to learn …