Howto: Download packages and dependencies for offline installation

From my perspective, offline installation is the weakest spot in the Linux superstructure. I understand the shared libraries concept, and I think it’s a marvelous thing, but it doesn’t erase the fact that if you’re not in a position for at least intermittent Internet access, it’s a dealbreaker.

Dependencies are almost impossible to track, and are usually scattered. The more complex a program is, the more dependencies it will have. Furthermore, those dependencies will probably have dependencies, and those have to be macromanaged too, if you can’t rely on aptitude to handle it for you.

There’s an added problem when you realize that an offline machine can’t be sure what versions are new. Until aptitude synchronizes with the repositories, the packages in its database are possibly obsolete.

In this solution, the idea is to create a web page that links to the Ubuntu Package Search pages, which can be opened on a machine that’s connected, and download the software plus its dependencies without too much hassle. From there, the packages can be ferried back to the offline box and installed manually with dpkg.

The first stumbling block to the concept is that there is no way to hard link straight to a deb package, since the offline machine can’t build links straight to it. The version number is unpredictable, and changes frequently within each release. Even more perplexing is the indexing system used on the repositories, where software is split alphabetically into subdirectories. If a package were to move, or the software to change its name, it’s altogether possible that the package might shift to a new location. That, again, is something an offline machine can’t foresee.

However, there is a mechanism on the search pages that can help. There is a built-in search option for software, accessible by tacking the package name on to the end of the URL. For example, to search for the leafpad package, you only need enter “http://packages.ubuntu.com/leafpad” into the address bar. The search mechanism with parse that automatically, and give you links to the software page, for each Ubuntu release.

So this method queries apt-cache on an offline machine with the depends option. The output is redirected back into apt-cache depends, then again for a third level of dependencies. Hopefully this will catch any oddball packages you might need.

apt-cache‘s output is rearranged into a workable list with sort, then filtered with a short (and probably clumsy) mawk program that cuts out the gunk and leaves you with a sparse html document that links to the package search pages.

From there, you’re within three clicks of the actual deb file, and once you have all the dependencies downloaded, you can transfer them back to your offline machine. Invoke them with dpkg -i *, and barring incident, it should install the software you asked for, without too much difficulty.

This method is not foolproof. I’ve tested it twice now, once with leafpad and its dependencies, and once with Critical Mass. Both times it worked fine. I also tested it with xorg, although I admit I didn’t ferry all of xorg’s dependencies back to my offline machine. There were just too many, and by then I was fairly confident it was working.

Anyway, it’s altogether possible that you’ll get a version mismatch, or a dependency that’s even deeper than three levels, that you didn’t download on the first try. I’m sorry. Just rerun the script using that package as the target, and after installing that software and its dependencies, your first package should work fine. Lather, rinse, repeat.

This isn’t much use installing software that doesn’t show up in apt’s repositories to start with. In other words, you can’t install Critical Mass out of universe if you haven’t synced your online machine with universe. Apt won’t know what universe is, doesn’t know Critical Mass is in it — for that matter, doesn’t even know Critical Mass exists. (The workaround to this is to download the metapackage for a game or an application, and dpkg it on the offline machine. That should tell apt what dependencies it needs, and from there you can run the script. … Does that make sense? :roll: )

This also won’t help much with upgrades or dist-upgrades, since again, a true offline machine has no idea whatsoever of what’s been improved or rereleased since it was born. If you want to meld your own system for handling upgrades and dist-upgrades, be my guest.

Lastly, I feel I should mention that neither the script and mawk program uses sudo, nor does either program use any software beyond an initial installation. In other words, you should be able to access apt-cache, sort and mawk from any Ubuntu machine … unless you threw out one of those packages. All of this — the script, for what it’s worth, and the program — are GPL 3.0, and you’re welcome to use it in your own Ubuntu adventures, however you see fit.

deps-script.sh Ubuntu pastebin version
PKGNAME=leafpad

apt-cache depends $PKGNAME | xargs apt-cache depends 2>/dev/null | xargs apt-cache depends 1>$PKGNAME-deps.txt 2>/dev/null
sort -d -u -b -o $PKGNAME-deps-sorted.txt $PKGNAME-deps.txt
mawk -f deps2html.mawk $PKGNAME-deps-sorted.txt > $PKGNAME-links.html

deps2html.mawk Ubuntu pastebin version
# deps2html.mawk
# 24 July 2007 by K.Mandla
# Released under GPL 3.0
# See http://www.gnu.org/licenses/gpl-3.0.txt for information and terms of the license.

# This is a very simple mawk program that sifts through the output of
# apt-cache depends, and builds a hyperlinked page that can be useful
# for installing software on offline machines. Of course, it's only
# practical to a point, since there's no way of hotlinking to a .deb
# package (because of the naming convention) and any offline machine
# is probably going to have an outdated package list anyway.

{
# First take out all the spaces, since that interferes with the creation
# of hyperlinks. Four spaces is more than enough, since it's very rare
# that more than four occur together, and if they do, they generally
# are in a line that will be removed anyway.

for (i = 1 ; i = 5 ; i++) gsub(" ","")

# Screen out anything marked as a dependency, since it's lower than
# we're willing to go. Also cut other marked packages, since those are
# either irrelevant or just in the way. The echo command is really only
# a dummy, kicking mawk to the next record without doing anything.
# There are probably better ways to do that.

if (match ($0,"Depends:") || match ($0,"Conflicts:") || match ($0,"Predepends:") || match ($0,"Suggests:") || match ($0,"Replaces:") || match ($0,"Recommends:") || match ($0,"Breaks:")) echo

# If it's a valid package and a reasonable dependency, build the link.
else print "[a href=\"http://packages.ubuntu.com/"$0"\" /]"$0"[/a][br /]"
}

Step by step: First, download or copy-and-paste the script and program into a file. Please note that you’re not required to use the script, although it does offer a quick way to get the entire business done. Otherwise, you’d have to type in some commands, and nobody likes to use the command line. I mean … er … yeah. Anyway.

If you copy and paste the mawk program, change the brackets in the final line to greater-than and less-than symbols. Once again, WordPress’s coding is in the way of my coding. :evil: If you download the script and program, cut off the .doc extension and add the dot back to where it ought to be — .sh or .mawk. Once again, WordPress’s ridiculous filename requirements are in the way of my blogging. … :evil:

Next, move them to the offline machine. Edit the script to change the PKGNAME variable to the name of your program, then run it with sh deps-script.sh. It will do all the hard work, and you’ll be left with three new files: PKGNAME-deps.txt, which is the raw output from apt-cache; PKGNAME-deps-sorted.txt, which is the output after the sort pass; and PKGNAME-links.html, which is a headerless, bodyless list of dependencies, coded as html links, that form a very simple Web page when opened in a browser.

Move the links file to an online machine — any type, any architecture, any operating system — and open it in a browser. Follow each link, picking your release and architecture, and finally following the download link. Download all your software packages, and ferry them back to the offline machine.

From there, you should be able to install your software manually, with sudo dpkg -i *, or a variation of those commands. This part is where you’re on your own, and if you’re lucky (type with your fingers crossed), your software should drop into place without a hitch. Ta-da! You just installed software on an offline machine, courtesy of apt-cache and mawk.

I hope this works for you. I know the offline-installation issue isn’t a huge deal for most people, now that we’ve entered a golden age of the Internet. Still, it’s always possible that someone, somewhere — like me — is working in an offline environment, and needs a way to get an application installed without access to the ‘Web. If it works, let me know. If it doesn’t, let me know. Cheers and good luck.

P.S.: No making fun of my programming skills. I hit the pinnacle of my programming career when I typed “Hello World” into my C-64. So just cool it. :twisted: On the other hand, if you have honest suggestions, I’m always willing to learn … :mrgreen:

About these ads

14 Responses to “Howto: Download packages and dependencies for offline installation”


  1. 1 Gregor 2007/08/27 at 10:05 PM

    This could be interesting for you: http://aptoncd.sf.net

  2. 2 K.Mandla 2007/08/27 at 10:33 PM

    Thanks Gregor. I had seen APTonCD a couple of months ago, but I decided it wasn’t what I wanted, mostly because it comes with a heap of Gnome dependencies. I think if it was a little more lightweight, it would be more useful to me. But thanks for the tip, I appreciate it. ;)

  3. 3 Jeremiah 2008/08/16 at 11:17 AM

    Hey there, I know this is an old post, but I thought you may be interested to know that I also implemented something like this in python. Code is at http://launchpad.net/pd , dev version is hosted on my computer at http://packagedepends.homelinux.org:8080 .

    At the moment, mine just calls aptitude download and apt-get build-dep, which I _think_ should satisfy the dependencies for most packages, but I’m not sure. I’ve been pondering doing something closer to your solution here (although, I’ll probably be working directly with the files in /var/lib/apt/lists to parse the dependency chain if I do), though I’m wondering how to deal with when to stop looking down the chain. I was thinking of maybe letting the user choose what ubuntu “flavor” they run, and then comparing dependencies to what’s in the flavor-desktop package definition, maybe building a set of what’s in the base on app-startup (a common set, plus an additional flavor-specific set).

    • 4 Manuel Bärenz 2011/04/09 at 5:55 AM

      Take care: apt-get build-dep only builds the dependencies missing on the downloading machine, not necessarily the ones missing on the machine you want to install on.

  4. 5 Ai 2009/05/20 at 12:16 PM

    I just wanted to let you know I’m working in an offline environment and having a heck of a time getting online. It’s nice to know I’m not alone! ^_^

  5. 6 Neil L 2009/09/18 at 7:30 AM

    You can also try using:
    apt-get build-dep\
    –download-only\
    –reinstall\
    –no-install-recommends\
    –yes\
    -o Dir::Etc::SourceList=sourcelistfilename
    -o Dir::Cache::Archives=foldertostorepackagesin
    listofpackages

    followed by the same command with “install” instead of “build-dep”

    The first command will get all of the dependencies for the packages in listofpackages from the repositories specified in sourcelistfilename and put them in the folder specified by foldertostorepackgesin. Make sure you use a / at the beginning of foldertostorepackagesin or you will also need to set the Dir::State option. The second command will get the actual packages. You need reinstall to make sure it pulls the packages even if they are already on the system you are using to get them. The –yes makes it so that you don’t have to answer yes about downloading each package. You may want to remove the –no-install-recommends depending on whether or not you want to also install recommended packages. I am not sure if build-dep is recursive but my assumption is that it is.

  6. 7 Theodore Biryukov 2009/12/16 at 5:51 PM

    There is else a solution.
    To get list of needed packages (package+all the dependencies) use this command:
    sudo apt-get –print-uris –yes install PACKAGE_NAME | grep ^\’ | cut -d\’ -f2 >mydownload.txt
    And then you get a text file with links. In linux you need just to type (on computer with internet acess):
    wget –input-file mydownload.txt
    Or use a download manager with possibility of downloading all the files from a list of urls from a text file.

    This is on this page: http://www.webupd8.org/2009/11/get-list-of-packages-and-dependencies.html

    • 8 Theodore Biryukov 2009/12/16 at 6:26 PM

      Sorry. It doesn’t work with out connection to the internet… And you script also doesn’t work. It gives all the files in the output but file PACKAGE_NAME-links.html is ”. So…

  7. 9 xirius 2010/04/17 at 7:47 AM

    try keryx. it answers all the problems related to offline ubuntu installations here’s the link..

    http://keryxproject.org

  8. 10 Jay Sprout 2010/06/14 at 11:12 PM

    I was so happy because I thought this solved my offline problem – I just got a laptop a week ago and installed Linux on it. But this solution depends on (no pun intended) your ONline machine also using Linus, does it not? All the online computers I have access to are Windows.

    • 11 K.Mandla 2010/06/15 at 7:18 AM

      It’s been a while since I wrote this Jay, but if I understand your question right, it shouldn’t matter that your online machines are Windows-driven. The scripts only build a list of links to packages you can install, and so the output is just an html file.

      Move that to a Windows machine, open it in a browser and then follow each one through to a download source. From there, move the downloaded packages back to the offline machine via USB drive or floppy or what have you.

      Or at least, that’s how I remember it working. :|

  9. 12 Aditya 2010/09/24 at 5:57 PM

    Doesn’t the ” sudo aptitude –download-only install ” work ???

  10. 13 jm 2011/04/06 at 10:30 AM

    yea in the golden age of the internet it is a big concern if you ever have to work with an offline machine so i find this quite useful. i have a laptop with no wired nor wireless connection working under a certain version of ubuntu i want to use (strange enough it works on other versions) and i found some drivers on the net but they are source codes, so now i need the build-essential package


  1. 1 Mighty Linuxz » Howto: Download packages and dependencies for offline installation Trackback on 2007/11/20 at 11:58 PM

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




Welcome!



Visit the Wiki!

Some recent desktops


May 6, 2011
Musca 0.9.24 on Crux Linux
150Mhz Pentium 96Mb 8Gb CF
 


May 14, 2011
IceWM 1.2.37 and Arch Linux
L2300 core duo 3Gb 320Gb

Some recent games


Apr. 21, 2011
Oolite on Xubuntu 11.04
L2300 core duo 3Gb 320Gb

Enter your email address to subscribe to this blog and receive notifications of new posts.

Join 405 other followers

License

This work is licensed under the GNU Free Documentation License. Please see the About page for details.

Blog Stats

  • 3,963,764 hits

Archives


Follow

Get every new post delivered to your Inbox.

Join 405 other followers

%d bloggers like this: