The arrow of time

Ivan Voras' blog

pkgng - best thing since sliced bread!

FreeBSD (and BSDs in general) traditionally have source-based upgrades and installs which extends to the third party software collections - ports or pkgsrc and similar. This is all fine and offers unprecedended flexibility when tailoring system to specific needs, but sometimes this flexibility is less important than ease of use or time savings which can only be achieved with binary packages. Enter pkgng, the next-generation binary package management system by Baptiste Daroussin and others, which replaces the old-style ports and packages system.

Pkgng has a large palette of features which are simply done right - ranging from using sqlite as the database for storing (and even exchanging) package information, to a very flexible way of using (and creating!) additional package repos (of course, packages are built from existing ports), but the single most important feature which should immediately appeal to anyone who has used ports for a long-ish time is this: it has two-way recursive dependancy tracking.

This is best explained in an example: if you installed apache22 version X and later want to upgrade it to version Y but the package for version Y has a dependancy on a newer version of pcre, it will upgrade both apache22, pcre and all packages depending on pcre in one operation. This is a "matter of course" for Linux distributions but was generally a pain in the ass to achieve with ports. Now, pkgng will do it automagically.

Concerning package versioning, the long-term plans are to establish "stable" sets of packages. This is still a "rolling release" system in the sense that no changes will be backported to previously released sets of packages, but will ensure that a specific package set stays on the repo servers for a certain time - a year or so, which minimizes the need for massive upgrade operations. There are lot of things yet to be defined for pkgng, so this may also change in the future.

You can install pkgng either from ports (in the ports-mgmt/pkg directory) or from a classical old-style package file - it doesn't matter. To start using pkgng it is best to start from a clean system, without any packages installed, but it can also be achieved by converting existing installed packages/ports to the pkgng format by running pkg2ng.

The next step is to copy /usr/local/etc/pkg.conf.sample to /usr/local/etc/pkg.conf and edit the PACKAGESITE line to an existing repository. Examples of these are:

  • http://pkgbeta.freebsd.org/freebsd-9-i386/latest -- for i386
  • http://pkgbeta.freebsd.org/freebsd-9-amd64/latest -- for amd64

After that, the command "pkg update" will update the remote repository data, and everything will be set up for using pkgng.

A new package can be installed from the remote repo by using "pkg install $packagename", e.g.:

# pkg install vim-lite
Updating repository catalogue
Repository catalogue is up-to-date, no need to fetch fresh copy
The following packages will be installed:

Installing libiconv: 1.14
Installing gettext: 0.18.1.1
Installing vim-lite: 7.3.556_1

The installation will require 36 MB more space

10 MB to be downloaded

Proceed with installing packages [y/N]: y
libiconv-1.14.txz 100% 595KB 595.3KB/s 595.3KB/s 00:01
gettext-0.18.1.1.txz 100% 4955KB 991.0KB/s 825.1KB/s 00:05
vim-lite-7.3.556_1.txz 100% 5117KB 1.0MB/s 607.4KB/s 00:05
Checking integrity... done
Installing libiconv-1.14... done
Installing gettext-0.18.1.1... done
Installing vim-lite-7.3.556_1... done

... and a list of packages can be queried with "pkg info":

# pkg info
gettext-0.18.1.1               GNU gettext package
libiconv-1.14                  A character set conversion library
pkg-1.0.r4                     New generation package manager
vim-lite-7.3.556_1             Vi "workalike", with many additional features (Lite package)

Note that, since a proper database is being used, "pkg info" is finally a fast operation. The rest of the commands are named similary to their old-style counterparts, and the whole list can be viewed simply by running "pkg" without arguments.

That's it - happy pkgng'ing!

Update:

Pkgng cannot be used simultaneusly with old-style binary packages and with old pkg_* management tools, simply because everything changed: the package format is different and the way the installed packages are tracked is different (and much better). If the two systems are used simultaneusly, all sorts of unwanted and erroneus behaviour will happen: files will be overwritten, dependancies will not be tracked properly, etc. - in short, the two system will fight badly. If you start using pkgng, do not continue to use the pkg_* tools and ports management tools like portupgrade and portmaster.

However, pkgng can coexist with from-source builds of ports by adding the line WITH_PKGNG=yes to /etc/make.conf. This will cause the newly built and installed ports to be registered properly in the pkgng database. While this is an excellent (and very much needed) idea when there is a need for some specific flags and options in ports's builds, it requires special care when upgrading with pkgng.

#1 Re: pkgng - best thing since sliced bread!

Added on 2012-07-26T11:20 by hshh

Some bad experience.

1. run pkg2ng second time will not merge updated ports (old pkg system) to pkgng

2. latest pkg-1.0.r4, pkg2ng will move /var/db/pkg/* to /var/db/pkg.bak, so I can't use system pkg_* to manage.

#2 Re: pkgng - best thing since sliced bread!

Added on 2012-07-26T11:26 by Ivan Voras

Yes, the system is not designed to be used in parallel with the old ports/packages - it replaces it.

#3 Re: pkgng - best thing since sliced bread!

Added on 2012-07-26T13:59 by DES

You can still install ports from the ports tree with pkgng. Just make sure you have WITH_PKGNG=yes in /etc/make.conf. Perhaps pkg2ng should do that for you...

#4 Re: pkgng - best thing since sliced bread!

Added on 2012-07-26T14:52 by Ivan Voras

Thanks, DES! I'll update the post.

#5 Re: pkgng - best thing since sliced bread!

Added on 2012-07-26T23:15 by Catalin
I think I'm not following. You shouldn't use portupgrade/portmaster, but building ports from source is OK.. I'm taking there's a -yet- there: "You shouldn't use portmaster yet." It horrifies me to think I have to figure out what needs updating by hand.

#6 Re: pkgng - best thing since sliced bread!

Added on 2012-07-27T00:49 by Ivan Voras

It's very simple: you cannot use portupgrade, portmaster or any other tool which reads old-style package registration data or uses pkg_* tools for any work, because pkgng introduces completely new package metadata infrastructure. This may or may not change (I'm sure the authors of portupgrade and portmaster will accept patches :) ).

However, almost all of the functions these tools introduced have been reimplemented in the pkg utility. For example, "pkg version" will list package versions and indicate with the usual "<", "=", ">" symbols which packages need upgrading. Of course, pkg itself can only upgrade from pkgng package repositories.

#7 Re: pkgng - best thing since sliced bread!

Added on 2012-07-27T00:55 by Ivan Voras

... continuing with the theme: "pkg autoremove" will remove "leaf" packages, installed as dependancies which are no longer needed, "pkg search" will search the repository for packages whose name contains the given string, "pkg which" queries local package metadata to find which package installed the given file, and "pkg install", when given a package name of an already existing package, will upgrade the specified package. pkgng also doesn't do "dumb" upgrades via the "remove-then-add" mechanism which was previously used, but allows packages to (optionally) perform intelligent, scripted upgrades.

#8 Re: pkgng - best thing since sliced bread!

Added on 2012-07-27T09:35 by Marc
You can use portmaster with pkgng with a patch from https://github.com/pkgng/pkgng/tree/master/ports. Store it in the ports files subdirectory and rebuild portmaster. Portupgrade user should use port-mgmt/portupgrade-devel

#9 ts lk 1993 ll vr gn

Added on 2012-10-10T04:47 by zzz

Smtms ts jst bst t dmt dft nd mv n. BSD s rll lttl mr thn nthr n f th nc-prmsng hs-bn OS' lk AmgOS nd BOS tht wr bndnd b rsnbl flk lng g. If wnt n pn-src UNIX OS tht ds vrthng lrd jst chs frm n f th mn Lnx flvrs tht sts r tst. Thr s n shm n fghtng th fght thn swtchng sds nc t s clr whch sd hs wn.

#10 Re: ts lk 1993 ll vr gn

Added on 2012-10-16T01:46 by null

ZZZ: I'll use an OS that lets me use vowels for now.

Pormaster has been updated to use pkg ng, a make config will expose the new option!

Hooray for that, while packages are fine, sometimes it's nice to have the sources around to patch for new features or fixes.

#11 Re: pkgng - best thing since sliced bread!

Added on 2012-10-16T13:52 by Marko Lerota

What about options in install? Can we choose them or not like in portmaster/portupgrade? For example if I want to install php53 can I say that I want apache module as well?

#12 Re: pkgng - best thing since sliced bread!

Added on 2013-12-16T14:00 by Pol hallen

Wonderful tool! Good work, thanks!

Missed total time when I install some packages.

Pol

Post your comment here!

Your name:
Comment title:
Text:
Type "xxx" here:

Comments are subject to moderation and will be deleted if deemed inappropriate. All content is © Ivan Voras. Comments are owned by their authors... who agree to basically surrender all rights by publishing them here :)