Helping you drive digital innovation
Subscribe
RSS Feed of the Mendix Blog
Thanks for Subscribing

Keep an eye out for Mendix resources coming straight to your inbox.

Creating and Using Debian Backports

on November 20, 2012

Share:

Introduction

At Mendix, we use the  Debian GNU/Linux operating system a lot. This distribution provides a stable Linux kernel and GNU userland, with a very broad collection of  additional software, maintained by a large community of software developers.

Production systems usually run on the stable release of the operating system. The latest release is called ‘stable’ because it must not change much after the release. Only security fixes and fixes for severe bugs (e.g. bugs causing data corruption) are allowed in, but no new versions of included software. When running package upgrades on a stable system, this ensures updated packages will never break existing configurations and API’s.

Preparation of the next stable Debian release happens in the  unstable and testing releases. When the next release will happen, stable gets renamed to oldstable, and testing gets renamed to stable. At the time of writing, the stable release is Debian 6, which has the code name ‘Squeeze’. The next stable release will be Debian 7, named ‘Wheezy’. The usage of the term unstable does not mean software available in the unstable part of Debian will crash more often, it just means there will be a lot of changes going on.

Backporting a package

So, stable does not change. But, in some cases, it might be desirable to run a newer version of software than included in our stable systems. Instead of upgrading the whole system to testing or unstable, it is better to rebuild a single software package from the testing or unstable part of the Debian archive to make it fit in the current stable release. This process is called creating a backport.

If the newer version of the software we want to use is available in testing or unstable, we can cherry pick that package and rebuild it on a system that runs stable, using the library dependencies and compiler that are available right there. When lucky, the package can rebuild without changes. If not, these issues need to be resolved, which can potentially become complicated. The package might for example have dependencies on specific newer versions of other packages that also aren’t available in the stable release.

Package maintainers of Debian can share  backports of their packages using the official Debian package archive, in case they’re  properly maintained and interesting for a large user base. An example of a package with a well maintained backport is the web server Nginx:

packages-nginx

There might be different reasons to use newer versions of software than is available in stable. In the case of Nginx, it might be newer features we’d like to use. Another example is creating the possibility to do more controlled upgrades to a new release. For example: during the upgrade from the previous stable release (Debian 5, Lenny) to the current one, we used backports of PostgreSQL packages to first do a full round of database-upgrades from PostgreSQL 8.3 to 8.4. After this was completed, we did the actual upgrade of all operating system packages in a separate maintenance schedule, during which the PostgreSQL package got replaced by the exact same package from which the backport was made.

An example: iftop

Today, I’ll use an easy example to demonstrate how to build a backport ourselves. Our victim package is a little tool called iftop. The name is a combination of “if” (interfaces) and “top” (a program to see the most active programs on the system). The iftop program shows a “top”-like view on network connections, also showing how much bandwidth they’re using:

iftop41

The system shown here is a router, routing IP packets in a stateless mode. Connection tracking is disabled on forwarded traffic. To get a view on the current flow of traffic through the router, we cannot use a tool like netstat, because netstat only shows connections to the router itself. Because connection tracking is disabled, we also cannot inspect the connection tracking table, using e.g. the iptstate or conntack programs.

In this case, iftop is a great tool to run diagnostics and get a view on the current traffic at this spot in the network. It will not only show which hosts are communicating, but also the amount of bandwidth that is used. By looking at the live traffic, iftop looks for packets with matching addresses and port numbers and presents a live view on what is happening.

But! Things are missing. We’re fooled by the output of the program because it does not show IPv6 traffic. According to the changelog of iftop, which we can find on the  homepage of the project support for IPv6 was added in a newer version, which is to be included in the next stable Debian release.

Because the network where our example router is located is fully IPv6-enabled, I’d like to have a newer iftop which can show me IPv6 right now!

When doing a search for iftop on packages.debian.org to show which versions of the software are already packaged in different releases, we can see iftop 1.0~pre2 is planned to be released with the next stable release of Debian. So, while using Debian stable for our production systems, we can try to make a backport of this package.

packages-iftop

There’s a bunch of things you need to consider when backporting, but this case is an easy example. It’s a tool we use for diagnostic purposes, so it’s not likely we’ll be introducing new risks for customer production processes. The new iftop version does not use any API that is not present in squeeze, and its dependencies do not conflict with the packages in stable. It just implements more functionality to read information that was already available.

Hands on!

For the package rebuild, I’ll be using the pbuilder program, which is a helper to build debian packages in a clean build environment. Pbuilder can create a chroot in which a minimalistic installation is placed of the target release we want to build on. After creating an archive file containing this base installation, pbuilder will extract the minimalistic system for every package build action, try to install build dependencies, create the package, and throw away all of it when it’s done. Well, except for the packages we get as a result of course! Because all work is done in the disposable chroot, the system itself does not get cluttered with all sort of installed packages which were needed as build dependencies.

Detailed information about the use of pbuilder can be found at:

To use pbuilder, we need to install it, together with a few extra tools we need to prepare the source package we’re going to feed to it.

# apt-get install --no-install-recommends pbuilder devscripts dpkg-dev build-essential fakeroot debhelper

The configuration file of pbuilder specifies the parameters based on which the build environment will be set up. In this example, I use a specific configuration file for my squeeze pbuilder setup.

~ 0-$ cat ~/.pbuilderrc-squeeze
DISTRIBUTION=squeeze
COMPONENTS="main contrib non-free"

MIRRORSITE=http://ftp.nl.debian.org/debian
DEBSECSERVER=http://security.debian.org/debian-security
OTHERMIRROR="deb ${DEBSECSERVER} ${DISTRIBUTION}/updates ${COMPONENTS}"

BUILDPLACE=/home/knorrie/pbuilder/build
BASETGZ=/home/knorrie/pbuilder/base-squeeze.tgz
BUILDRESULT=/home/knorrie/pbuilder/result
APTCACHE=/home/knorrie/pbuilder/apt-cache

DEBBUILDOPTS="-sa"

if [[ -n $SUDO_USER ]]; then
  BUILDRESULTUID=$SUDO_UID
  BUILDRESULTGID=$SUDO_GID
fi

Creating the build chroot needs only to be done once. In order to do so, pbuilder needs root permissions, because the chroot will be like a real system with different user accounts in it. For building packages, I use a separate development system, running wheezy, where my user account has permissions to use sudo to execute pbuilder.

~ 0-$ sudo pbuilder create --configfile ~/.pbuilderrc-squeeze
[...]
I: Current time: Mon Nov 19 00:13:47 CET 2012
I: pbuilder-time-stamp: 1353280427
I: Building the build environment
I: running debootstrap
/usr/sbin/debootstrap
I: Retrieving InRelease
I: Failed to retrieve InRelease
I: Retrieving Release
I: Retrieving Release.gpg
I: Checking Release signature
I: Valid Release signature (key id 0E4EDE2C7F3E1FC0D033800E64481591B98321F9)
I: Retrieving Packages
[...]
I: creating base tarball [/home/knorrie/pbuilder/base-squeeze.tgz]
I: cleaning the build env
I: removing directory /home/knorrie/pbuilder/build/3106 and its subdirectories

When using pbuilder update instead of create, it’s possible to refresh the archived chroot with the latest package updates.

Next step is to get the source package of iftop from testing and adjust it for building on stable. On the package overview page of iftop at packages.debian.org, there’s a link to the file iftop_1.0~pre2-3.dsc, which we can use to download the source package and extract the contents:

~/src/deb-src 0-$ dget http://ftp.de.debian.org/debian/pool/main/i/iftop/iftop_1.0~pre2-3.dsc
[...]
~/src/deb-src 0-$ dpkg-source -x iftop_1.0~pre2-3.dsc
[...]
dpkg-source: info: extracting iftop in iftop-1.0~pre2
dpkg-source: info: unpacking iftop_1.0~pre2.orig.tar.gz
dpkg-source: info: applying iftop_1.0~pre2-3.diff.gz
~/src/deb-src 0-$ cd iftop-1.0~pre2/
~/src/deb-src/iftop-1.0~pre2 0-$

First of all, we now read the changelog (at iftop-1.0~pre2/debian/changelog) of the package, to see if there are any hints about changes that are backwards incompatible with the previous release, which would make our backported package fail to build or run.

~/src/deb-src/iftop-1.0~pre2 0-$ less debian/changelog

We check the control file to see if there’s any dependency that could not be resolved on squeeze, which does not seem to be the case.

~/src/deb-src/iftop-1.0~pre2 0-$ cat debian/control
Source: iftop
Section: net
Priority: optional
Maintainer: Alexander Reichle-Schmehl <tolimar@debian.org>
Build-Depends: debhelper (>= 7.0.50~), autotools-dev, libncurses5-dev, libpcap0.8-dev
Standards-Version: 3.9.3
Homepage: http://www.ex-parrot.com/~pdw/iftop/
Vcs-Svn: svn://svn.debian.org/svn/collab-maint/deb-maint/iftop/trunk/
Vcs-Browser: http://svn.debian.org/viewsvn/collab-maint/deb-maint/iftop/

Package: iftop
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}
Description: displays bandwidth usage information on an network interface
 iftop does for network usage what top(1) does for CPU usage. It listens to
 network traffic on a named interface and displays a table of current bandwidth
 usage by pairs of hosts. Handy for answering the question "Why is my Internet
 link so slow?".

[...]

Next step is to edit the changelog file. Our backported package will get a version number that is lower than the version number the package has in testing, because it must be replaced with the actual package we’re backporting from when fully upgrading to the new stable release later. Instead of randomly decreasing numbers in the version string, the tilde (~) character is used in the  debian version logic, which has a magic value of ‘less than nothing’. Actually, it seems this trick has already been used for this package. The version in testing is 1.0~pre2, which means an upload of version 1.0 will make the package upgrade from 1.0~pre2 to 1.0. This is of course the desired behaviour, as using a version string like 1.0-pre2 would render that impossible, because it’s higher than 1.0.

The version number I’m going to use is 1.0~pre2-3~mxbp60+1, as it’s the first backport revision of a ‘mendix backport’ of 1.0~pre2-3 to Debian 6.0. Don’t use a hyphen instead of the plus sign, because that will mess up the semantics of your version string. Explaining the reason why is left as an exercise to the reader.

~/src/deb-src/iftop-1.0~pre2 0-$ dch --bpo
[...]
~/src/deb-src/iftop-1.0~pre2 0-$ head -n 11 debian/changelog
iftop (1.0~pre2-3~mxbp60+1) squeeze-backports; urgency=low

  * Rebuild for squeeze-backports.

 -- Hans van Kranenburg <hans.van.kranenburg@mendix.com>  Mon, 19 Nov 2012 00:20:23 +0100

iftop (1.0~pre2-3) unstable; urgency=low

  * Enable hardening buildflags in debian/rules (Closes: #675849)

 -- Alexander Reichle-Schmehl <tolimar@debian.org>  Tue, 22 May 2012 16:35:48 +0200

Now, we can use pdebuild to let pbuilder do it’s work. A source package will be created using our current changes, which will be put into an extracted copy of the chroot, in which the package will be built:

~/src/deb-src/iftop-1.0~pre2 0-$ pdebuild --configfile ~/.pbuilderrc-squeeze
[...]
dpkg-source: info: building iftop using existing iftop_1.0~pre2.orig.tar.gz
dpkg-source: info: building iftop in iftop_1.0~pre2-3~mxbp60+1.diff.gz
dpkg-source: info: building iftop in iftop_1.0~pre2-3~mxbp60+1.dsc
[...]
dpkg-buildpackage: source package iftop
dpkg-buildpackage: source version 1.0~pre2-3~mxbp60+1
dpkg-buildpackage: source changed by Hans van Kranenburg <hans.van.kranenburg@mendix.com>
 dpkg-source --before-build iftop-1.0~pre2
dpkg-buildpackage: host architecture amd64
 fakeroot debian/rules clean
dh clean
dh: Sorry, but 8 is the highest compatibility level supported by this debhelper.
make: *** [clean] Error 9
dpkg-buildpackage: error: fakeroot debian/rules clean gave error exit status 2
E: Failed autobuilding of package

Whoa! The build fails. A little research on the differences between recent versions of the packaging reveals the package maintainer bumped the debhelper compatibility level (the number in the debian/compat file) from 7 to 9, which debhelper on Squeeze does not know of. This was done in the most recent change of the package, when enabling  hardening buildflags, which is a release goal for Wheezy.

diff -ruN /home/knorrie/blaat/iftop-1.0~pre2/debian/compat debian/compat
--- /home/knorrie/blaat/iftop-1.0~pre2/debian/compat    2012-11-19 01:30:26.000000000 +0100
+++ debian/compat   2012-11-20 00:19:57.716392189 +0100
@@ -1 +1 @@
-7
+9
diff -ruN /home/knorrie/blaat/iftop-1.0~pre2/debian/rules debian/rules
--- /home/knorrie/blaat/iftop-1.0~pre2/debian/rules 2012-11-19 01:30:26.000000000 +0100
+++ debian/rules    2012-11-19 00:22:51.000000000 +0100
@@ -1,5 +1,9 @@
 #!/usr/bin/make -f

+export DEB_BUILD_MAINT_OPTIONS = hardening=+all
+export DEB_CFLAGS_MAINT_APPEND  = -DNO_SYSTEM -Wall -DUSE_GETNAMEINFO=1 -DUSE_GETIFADDRS=1
+export DEB_LDFLAGS_MAINT_APPEND = -Wl,-z,defs
+
 %:
    dh ${@}

@@ -10,9 +14,7 @@
 override_dh_auto_configure:
    [ -r /usr/share/misc/config.guess ] && cp -f /usr/share/misc/config.guess config/config.guess
    [ -r /usr/share/misc/config.sub ] && cp -f /usr/share/misc/config.sub config/config.sub
-
-   dh_auto_configure -- LDFLAGS="${LDFLAGS} -Wl,-z,defs" \
-       CFLAGS="${CFLAGS} -DNO_SYSTEM -Wall -DUSE_GETNAMEINFO=1 -DUSE_GETIFADDRS=1"
+   dh_auto_configure

 override_dh_auto_install:
    dh_auto_install -- DESTDIR=$(CURDIR)/debian/iftop

Using this newer compatibility level enables additional functionality which helps building the package with more secure compile options. Actually, the maintainer forgot to also bump the debhelper build dependency to a version that properly supports this. When rebuilding the package on Squeeze, dependencies seem to be resolved, but the package fails to build.

Besides reporting this as a bug, there’s a few options we have right now:

  1. Build the package using additional backported packages of debhelper and dpkg-dev, which are available as ‘official’ backports, so we don’t need to create them ourselves
    Or..
  2. Revert the last change, which is essentially the same as trying to backport from the previous debian revision of 1.0~pre2, which is 1.0~pre2-2, instead of 1.0~pre2-3

I’m choosing option 2. Package version 1.0~pre2-2 is available on snapshots.debian.org, so we can repeat the procedure of getting the source package, extracting it, adding a changelog item to set the backport package version and use pbuilder to build the package…

~/src/deb-src 0-$ dget http://snapshot.debian.org/archive/debian/20120522T223557Z/pool/main/i/iftop/iftop_1.0~pre2-2.dsc
~/src/deb-src 0-$ dpkg-source -x iftop_1.0~pre2-2.dsc
~/src/deb-src 0-$ cd iftop-1.0~pre2/
~/src/deb-src/iftop-1.0~pre2 0-$ dch --bpo
~/src/deb-src/iftop-1.0~pre2 0-$ head -n 5 debian/changelog
iftop (1.0~pre2-2~mxbp60+1) squeeze-backports; urgency=low

  * Rebuild for squeeze-backports.

 -- Hans van Kranenburg <hans.van.kranenburg@mendix.com>  Tue, 20 Nov 2012 01:27:44 +0100
~/src/deb-src/iftop-1.0~pre2 0-$ pdebuild --configfile ~/.pbuilderrc-squeeze
[... magic happens here ...]
dpkg-deb: building package `iftop' in `../iftop_1.0~pre2-2~mxbp60+1_amd64.deb'.
dpkg-deb: building package `iftop-dbg' in `../iftop-dbg_1.0~pre2-2~mxbp60+1_amd64.deb'.
 dpkg-genchanges -sa >../iftop_1.0~pre2-2~mxbp60+1_amd64.changes
dpkg-genchanges: warning: the current version (1.0~pre2-2~mxbp60+1) is earlier than the previous one (1.0~pre2-2)
dpkg-genchanges: including full source code in upload

Yay! Here we got our iftop, version 1.0~pre2-2~mxbp60+1, built as backport for stable.

~/src/deb-src/iftop-1.0~pre2 0-$ ls -l ~/pbuilder/result/
total 284
-rw-r--r-- 1 knorrie knorrie   1997 Nov 20 01:30 iftop_1.0~pre2-2~mxbp60+1_amd64.changes
-rw-r--r-- 1 knorrie knorrie  40104 Nov 20 01:30 iftop_1.0~pre2-2~mxbp60+1_amd64.deb
-rw-r--r-- 1 knorrie knorrie   5185 Nov 20 01:30 iftop_1.0~pre2-2~mxbp60+1.diff.gz
-rw-r--r-- 1 knorrie knorrie   1005 Nov 20 01:30 iftop_1.0~pre2-2~mxbp60+1.dsc
-rw-r--r-- 1 knorrie knorrie 172736 Nov 20 01:18 iftop_1.0~pre2.orig.tar.gz
-rw-r--r-- 1 knorrie knorrie  54670 Nov 20 01:30 iftop-dbg_1.0~pre2-2~mxbp60+1_amd64.deb

Concluding…

Backporting Debian packages from the testing release to make them run on stable can be done for several reasons, like a desire to use newer versions of software available in stable or for doing release-upgrades in multiple steps. It’s certainly a fun way to learn more about Debian and the packaging system. Not all packages that will be available in the next release, and are in testing now, will rebuild on stable without making or reverting changes that need manual resolving.

The pbuilder tool is a very valuable tool that can do all of the complicated work involved in building an actual package in a clean, consistent and reproducible way.

What’s next?

There’s more use for pbuilder instead of only making backports, like maintaining a custom version of a package, while adding specific patches using the quilt tool to organize patches that are made to the upstream version of a package.

After successfully building the package, it can be digitally signed with a GPG key and be uploaded to a package repository, after which the package can be installed using apt-get, just like we would do with any other debian package. Actually, that’s what I did when creating this backport some time ago for use by Mendix. A tool I recommend for this job is  reprepro which can manage setting up a complete repository, while hiding a lot of complexity we don’t need to specifically know of when just getting started, just like pbuilder does.

In the end, I of course installed the package from the repository onto the router mentioned above, which now shows IPv6 traffic when using iftop!

iftop6

Tags:

Subscribe to Our Blog

Receive Mendix platform tips, tricks, and other resources straight to your inbox every two weeks.

RSS Feed of the Mendix Blog

About Hans van Kranenburg

Hans is a senior sysops @ mendix. He loves Debian and extending the Intarwebz by building computer networks.