When creating software, it’s often easy to package up software in a convenient installation format for the end-user. On Debian / Ubuntu based systems, this is often done by creating a .deb package that the end-user can install. Unfortunately, there is not much information on the web about how exactly to do this, or what information there is can sometimes be spread out over many different sites. This tutorial attempts to remedy that. The packaging is simple once you understand how to do it.
Note that this tutorial does not go over how to officially package software for Debian / Ubuntu; there are project policies that you must follow for that to be the case(see the Debian Policy Manual). This tutorial does attempt to follow the packaging guidelines as much as possible in order to provide a good starting point.
There are a number of tools that claim to help you package up software(such as fpm or CPack deb among others), however it is my experience that these tools are not always good at packaging software up correctly. Moreover, they attempt to hide complexity but add their own layer of complexity is not needed when you know how to properly package software.
All of the example projects that are used in this tutorial may be found on Github.
Packaging Basics
A Debian package is at its core an archive file that has a specific structure of files that gets extracted to the root directory of your system. Since the files in the archive are in the proper structure, they will get installed in the correct directory when the archive is extracted(e.g. /usr/bin, /usr/include, etc).
File Naming
When you download a deb file from an APT repository, you may have noticed that the deb file name looks something like this:
apt_1.8.2.2_amd64.deb
There is a lot of information encoded in the filename, so lets split it up and take a look at it. The first part (apt) is the name of the package. After the first underscore comes the version(1.8.2.2), and after the second underscore comes the architecture(amd64).
Versions
As we see in the above example, the version of apt is 1.8.2.2. Sometimes however, you may see packages with either a ‘+’ or a ‘~’ in the version like the following:
libpam-systemd_241-7~deb10u6_amd64.deb
distro-info-data_0.41+deb10u3_all.deb
These characters are special in the way that package versions are sorted. The Debian policy manual goes into more detail, but basically think of the ~ as being ‘before/prerelease’ and ‘+’ being ‘after’. In the above example, libpam-systemd is the prerelease version of 241-7, while distro-info-data is after version 0.41.
Buildsystems Supported
The packaging scripts are generally smart enough to know what buildsystem your package is using(autotools, cmake, qmake, etc). Because of that, there is generally not much that you need to do in order to customize your buildsystem.
When using CMake, make sure to include GNUInstallDirs to ensure that libraries will get installed in the correct folder for multiarch compatability(e.g. /usr/lib/x86_64-linux-gnu/)
Getting the Tools
In order to create Debian packages, you need to have some tools installed on your system that help with the package maintenance. You’ll want to install at lest the following packages:
$ sudo apt-get install dpkg-dev devscripts dh-make
Creating a simple package
The first package that we’re going to make and install will simply install an application to /usr/bin that we can run. This will also utilize CMake as the buildsystem.
First, our CMake configuration:
project( SimplePackage )
cmake_minimum_required( VERSION 3.10 )
add_executable( simple-application main.c )
#
# Configure our installation. Install to the proper bin directory
#
include( GNUInstallDirs )
install(TARGETS simple-application
RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
)
Now, our application:
#include <stdio.h>
int main( int argc, char** argv ){
printf( "Hello world from simple application!\n" );
return 0;
}
Note that the important part in the CMakeLists.txt is to make sure that you include GNUInstallDirs. This will configure CMake to install the application into the correct folder, be it /usr/local/bin or /usr/bin. Now, it’s time to create our packaging. In order to do that, we will use the dh_make application to create the basic structure of the debian/ folder. For the purposes of this tutorial, the only settings we need to worry about for dh_make are –single(to indicate that only one package is being made), a simple copyright license(in this case MIT), a maintainer email, native(we will discuss this more later), and the package name with the version. The command for dh_make is as follows:
$ dh_make --single --copyright mit --email email@example.com --native --packagename simple-package_1.0
As we can see, we now have a debian/ folder with the basic structure needed for the package. Many of the files that are in here are example files; as we don’t need these for this package, let’s remove all of those files for now:
$ rm debian/.ex debian/.EX debian/README* debian/*.docs
At this point we can now build and install the package. In order to do that, run the dpkg-buildpackage command:
$ dpkg-buildpackage -us -uc -b
One directory up, there will be a new .deb file. Let’s go and install that manually:
$ sudo dpkg -i ../simple-package.0_amd64.deb
Now that it has been installed, we can verify that it has been installed and on our PATH:
$ ls -l /usr/bin/simple-application
-rwxr-xr-x 1 root root 14440 May 10 17:50 /usr/bin/simple-application
$ simple-application
Hello world from simple application!
Congratulations! You have now made your first debian package! Now, let’s go into more detail as to what is actually in the debian/ folder.
Files in the debian/ folder
changelog – Changelog of the package. Controls the versioning of the package.
compat – The compatibility level of debhelper
control – Main packaging information for our package
copyright – Information about copyright of our package and the packaging scripts
rules – A Makefile that controls the building of the package.
source/format – The type of package that this is. Packages can be either native(they have debian packaging with them), or quilt(patches can be applied). Quilt would be used for packages that already have an upstream source that does not include debian packaging information.
These files are the most critical part of the entire packaging system, and control various aspects of how the package is built. For example, the control file contains the information about the name of the package, what dependencies are required, etc. For example, if we look at our list of currently installed packages, we notice that the description of our simple-application indicates that it needs to be filled in:
$ dpkg -l | grep simple-package
ii simple-package 1.0 amd64 <insert up to 60 chars description>
We can change this by modifying the debian/control file to say something useful in the ‘Description:’ field.
Now, when we build and install our package again, the new information will show up:
$ dpkg-buildpackage -us -uc -b
$ dpkg -i ../simple-package_1.0_amd64.deb
$ dpkg -l | grep simple-package
ii simple-package 1.0 amd64 A simple package
One other thing to do in our control file is to make sure that when we build the package, we have the correct packages installed. You can always assume that the build-essential package is installed for building. In our case, since we use CMake, we need to add that to our build-depends line:
Build-Depends: debhelper (>= 11), cmake
Dependencies for packages
As we have seen above, the build-depends are required for installing our build dependencies. What happens if we need to install a package for runtime support? Well, that’s where the ‘Depends:’ section comes into play for the package. You’ll notice that it already has replacement fields for libraries that may be needed:
Depends: ${shlibs:Depends}, ${misc:Depends}
This handles the automatic dependencies for the application. As long as the application is linked with a particular library(for example libpopt), then the package that contains the library will be automatically added to our depends so that it will be pulled in when the package is installed through apt.
If you need another package at runtime that you don’t link with(perhaps you need zip for some reason), you add that to the Depends: line in order to have it installed through apt:
Depends: zip, ${shlibs:Depends}, ${misc:Depends}
Building a library
So far, we have only built a simple application to install on the system. Now, it’s time to build a library that can be used by other applications on the system. One of the other important things about libraries is that they generally have two packages associated with them: one that contains the actual .so file for runtime support, and one -dev package that contains the headers that you would use to compile an application that uses the library.
As before, we will have a very simple library that we will install:
project( SimplePackageLibrary )
cmake_minimum_required( VERSION 3.10 )
add_library( simple-lib SHARED simple-lib.c )
set_target_properties(simple-lib PROPERTIES
VERSION 0
SOVERSION 0
)
#
# Configure our installation. Install to the proper bin directory
#
include( GNUInstallDirs )
install(TARGETS simple-lib
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
)
install(FILES simple-lib.h
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/simple-lib
)
The header and C file are trivial – view this part of the tutorial on Github for the actual implementation.
As before, we will use dh_make to create the initial skeleton for Debian packaging, except that we will now tell it that we are packaging a library:
$ dh_make --library --copyright mit --email email@example.com --native --packagename simple-package-library_1.0
$ rm debian/.ex debian/.EX debian/README* debian/*.docs
If we look into the debian/ folder that has been created, we see that we now have several new files:
simple-package-library1.dirs
simple-package-library1.install
simple-package-library-dev.dirs
simple-package-library-dev.install
We also got a warning from dh_make:
Make sure you edit debian/control and change the Package: lines from
simple-package-libraryBROKEN to something else, such as simple-package-library1
This is because we are now making more than one package from the same sources. Because this is a library that we are now packaging, we have a -dev package that contains the files needed to compile(the header files and some symlinks), as well as the runtime dependencies that are needed when you run an application that uses your library.
First, we will fix the debian/control file to be correct:
... standard section above ...
Package: simple-package-library-dev
Section: libdevel
Architecture: any
Multi-Arch: same
Depends: simple-package-library1 (= ${binary:Version}), ${misc:Depends}
Description: Simple package headers
Simple library package headers for development
Package: simple-package-library1
Architecture: any
Multi-Arch: same
Depends: ${shlibs:Depends}, ${misc:Depends}
Description: Simple package library(runtime)
Simple package libray runtime files
Now, let’s talk about what the .dirs and .install files do. Essentially, they tell the packaging utilities what files go into what package. For every Package: line that appears in the debian/control file, there should be a .dirs and .install file that indicate what needs to be installed. The defaults are sane default values, but do generally need to be tweaked. If we try to build the package as-is, you will see that we will get an error:
dh_install: Cannot find (any matches for) "usr/lib/*/pkgconfig/*" (tried in ., debian/tmp)
dh_install: simple-package-library-dev missing files: usr/lib/*/pkgconfig/*
dh_install: Cannot find (any matches for) "usr/share/pkgconfig/*" (tried in ., debian/tmp)
dh_install: simple-package-library-dev missing files: usr/share/pkgconfig/*
dh_install: missing files, aborting
This is because when the package is being made, there is nothing in these directories to install. To fix this, we will simply edit the *.dirs and *.install files as needed to remove the offending lines. In this case, we only need to edit the simple-package-library-dev.install file.
simple-package-library-dev.install before:
usr/include/*
usr/lib/*/lib*.so
usr/lib/*/pkgconfig/
usr/share/pkgconfig/*
simple-package-library-dev.install after:
usr/include/*
usr/lib/*/lib*.so
Now we can build and install our library package:
$ dpkg-buildpackage -us -uc -b
$ sudo dpkg -i ../simple-package-library-dev_1.0_amd64.deb ../simple-package-library1_1.0_amd64.deb
Utilizing the same technique as creating a library, we can also create many packages from a single source package simply by adding new packages to our debian/control file, along with the .dirs and .install file.
Pre-installation and post-installation scripts
When packages are installed or removed, it is sometimes required to do some setup(such as adding a daemon user to run an application) or remove data that is no longer needed. In order to do that, there are special scripts that will run at well-defined times of the installation process. These scripts are called preinst, postinst, prerm, postrm. As their names imply, they will run before installation, after installation, before removal, and after removal of the package. Up until now, we have been removing these scripts from the debian/ folder – if you need them back, you can find a copy on your system at /usr/share/debhelper/dh_make/debian
As a concrete example, if you need to add a daemon user to run an application, you should call ‘adduser’ in the ‘postinst’ script.
Installing a systemd service
Not everything that we are going to install is an application that a user will use directly. Sometimes, we need to install daemon applications. The modern way to run these applications is with systemd; fortunately, the packaging tools make this easy to do. As of debhelper compat level 10, the service will be run automatically when installed, so there is nothing special to do! You can check the compat version by looking at the debian/compat file.
Versioning packages and using dch
All of the packages that we have created up until this point have been at version 1.0, as that is the default version that we have set them to. Of course, no software stays static for a long period of time, and as such the version must be updated. The version of the package is handled in the changelog. Here is an example from our simple packaging application:
simple-package (1.0) unstable; urgency=medium
* Initial Release.
-- robert <email@example.com> Mon, 10 May 2021 17:50:35 -0400
The first line contains the package name, the version, and the distribution that this package is for. For example, the distribution could be one of buster, bullseye, etc. for Debian. Underneath that is the actual changelog of the package.
Normally, you do not want to edit this file by hand. Instead, you should use the dch program in order to make the changes, as they will be properly formatted. For example, let’s say that we are now working on version 1.1 of the software. We would then execute one of the following commands to add a new section to the changelog indicating the new version:
$ dch -i # Increment the version in a smart way. If your email is not set or different from the maintainer, will default to non-maintainer upload
$ dch -v 1.1 # Explicitly set the version to 1.1
dch can be used to edit other parts of the changelog. This is generally the tool that you should use to edit the changelog, instead of editing it by hand.
To add a new entry to the changelog, use dch without any arguments:
$ dch "Example changelog entry"
To mark the debian version as released(e.g. right before you tag):
$ dch -r
Packaging an upstream(3rd party) package
Up until now, we have been packaging ‘native’ packages. This is fine when we are packaging something that is controlled in the same repo as the debian packaging, and gets released with it. Many times however, we need to package third-party libraries that are created by other people. In this case, we will need to create a ‘quilt’ format package. With quilt packages, patches can be applied to the unmodified upstream sources.
The first thing we will need to do is to use pristine-tar to import the original upstream sources. Note that pristine-tar expects the original data to be ‘upstream’ branch, so let’s go and create that branch now:
$ git checkout --orphan upstream
$ rm -rf *
$ git commit -m "Rverting files for upstream"
Now that we have our sources all configured, properly, it’s time to use dh_make to create the basic debian packaging structure:
$ dh_make --single --packagename simple-thirdparty_1.0 --email user@example.com --copyright mit
$ rm debian/.ex debian/.EX debian/README* debian/*.docs
Similar to our simple packaging, we will need to edit the debian/control file to add in a useful description of the package. However, at this point we will now use quilt to edit a file to change what is printed out when the application is run.
Before we do that however, we will want to set a few environment variables so that quilt will put the patches in the correct folder. As seen in the Debian wiki, the easist thing to do is to simply create a ~/.quiltrc with the following information:
QUILT_PATCHES=debian/patches
QUILT_NO_DIFF_INDEX=1
QUILT_NO_DIFF_TIMESTAMPS=1
QUILT_REFRESH_ARGS="-p ab"
QUILT_DIFF_ARGS="--color=auto" # If you want some color when using quilt diff.
QUILT_PATCH_OPTS="--reject-format=unified"
QUILT_COLORS="diff_hdr=1;32:diff_add=1;34:diff_rem=1;31:diff_hunk=1;33:diff_ctx=35:diff_cctx=33"
The basic way to run quilt is to do the following:
$ quilt push -a # apply existing patches to source
$ quilt new 001-mypatch.diff # Create a new patch
$ quilt add main.c # add a file to the patch. A single patch can modify multiple files
At this point, you may now edit the main.c file. In this case, all we will do is to edit the text that gets printed out when the application runs. Once we have done that, finish the patch:
$ quilt refresh # Update the diff file
$ quilt pop -a # Unapply all patches
Once we have done this, our file debian/patches/001-mypatch.diff will look something like the following:
Index: thirdparty-package/main.c
===================================================================
--- thirdparty-package.orig/main.c
+++ thirdparty-package/main.c
@@ -1,6 +1,6 @@
#include <stdio.h>
int main( int argc, char** argv ){
- printf( "Hello world from simple application!\n" );
+ printf( "Hello world from simple application! Debian patched!\n" );
return 0;
}
At this point, you can now build the package and install as before. The difference now is that the patch(es) that we just made will now be applied to the source before the package is built. These patches can be used to fix bugs in a package without requiring those changes to be sent upstream, or alternatively to backport a fix that is already upstream but not in the specific version that we have just packaged.
See also the Debian documentation on using quilt.
Looking inside packages
At their core, Debian packages are just unix archives. Because of this, you can easily browse them using standard tools on your system. file-roller is one application that will provide you a GUI for browsing inside of packages.
Building packages with Jenkins
Once the packaging has been completed for a particular piece of software, setting up a way to build the package in a clean environment is a good idea. The official Debian builders use sbuild, however pbuilder also exists to help you build packages in a clean environment every time. The advantage of building in a clean environment means that the build is immune from “works on my system” problems.
In order to build with Jenkins, the easiest thing to do is to install the Debian Pbuilder plugin and follow the directions.
This plugin will setup pbuilder to create a clean environment for each build, and also lets you build packages for different architectures. For example, arm packages can be built on x86, albeit through emulation.