27. 10. 2023
Development
An RPM (RedHat Package Manager) package is the file format used by RHEL and CentOS, and their package manager yum (now called dnf). Since NetEye is based on CentOS, we use this standard package manager for distribution. How an RPM is constructed is defined in so-called spec files. In this blog post I’m going to show you a few things I’ve learned while working with spec files and RPMs over the last few years.
If you’re interested in the basics on how to start out with building RPMs, you can refer to the Fedora Docs.
Use %include for Common Macros in Different Spec Files
Early in the development of NetEye we noticed that related packages, for instance plugins for the same module, were often re-declaring the same paths in macros over and over again for each package. When one of these paths changed, every spec file in every module needed to replicate the change.
We solved this by using a functionality that was recently added to the RPM specification, namely the %include directive in spec files. For example, suppose that we want to package a module and allow the dynamic installation of plugins for this module packaged separately.
We can define macros which need to be used in each package in a shared .inc file, lets call it my_module.inc, containing the following definitions:%define module_dir %{_datadir}/my_module%define module_plugin_dir %{module_dir}/plugins
Then each spec file which needs one of these macros can include them using: %include ./ci/build/my_module .inc
From this point on, both the %{module_dir} and %{module_plugin_dir} macros will be available in the spec file as if they were defined directly inside it. Moving the entire application from %{_datadir}/my_module to, e.g. %{datadir}/neteye/my_module, just became a single line change.
Preview-Resolved Macros in Spec Files
Depending on what needs to be packaged, producing an RPM can take from a few seconds to up to more than an hour. Most of that time is usually taken up by the %build phase.
As we saw before, macros are handy tools to simplify your life as a packager, however when using many of them, it becomes very tedious to follow the process in your head, sometimes you don’t remember what macros like %{_localstatedir} or %{_datadir} refer to exactly.
When trying to get them right, the naive approach would be to run the build, and then fix whatever errors are thrown back.
A better strategy is to resolve all macros using the rpmspec command and then look at the output with resolved macros and real paths. The following command will print the resolved spec file content to STDOUT:
rpmspec -P my_module_plugin.spec
Let’s resolve this partial spec file content:
%define module_dir %{_datadir}/my_module
%define module_plugin_dir %{module_dir}/plugins
%define my_dir %{module_plugin_dir}/%{name}
Name: my_plugin
Source0: my_plugin.sh
[...]
%install
mkdir -p %{buildroot}/%{my_dir}
install -p -m 755 %{SOURCE0} %{buildroot}/%{my_dir}/
The output will be something similar to the following:
Name: my_plugin
Source0: my_plugin.sh
[...]
%install
mkdir -p /home/benjamin/rpmbuild/BUILDROOT/my_plugin-0.0.1-1.fc32.noarch/usr/share/my_module/plugins/my_plugin
install -p -m 755 /home/benjamin/rpmbuild/SOURCES/my_plugin.sh /home/benjamin/rpmbuild/BUILDROOT/my_plugin-0.0.1-1.fc32.noarch/usr/share/my_module/plugins/my_plugin/
While this is not very readable at first sight, it is exactly what is executed during the build. After a little time, you will get accustomed to recognizing typical packaging errors such as forgotten %{buildroot} macros or macros with a typo that have not been resolved, resulting visually in a much shorter line than expected.
Visually Navigating the RPM Dependency Tree
When trying to analyze dependency trees of RPMs it’s handy to use the --whatrequires and --requires flag of the rpm command. It’s fast and easy to see what happens in this first layer. However, when trying to understand how on earth some random package deep in the dependency tree finished up on your system, doing it by hand can be a bit tedious.
Cue rpmreaper. It’s a tiny interactive TUI tool provided by the epel repository which allows you to navigate up and down the dependency tree at will, as you can see in the following screenshot.
Starting from vim-common and going up, we can see that it is required by vim-powerline and vim-enhanced which by itself also requires vim-powerline.
Going down instead, we can interactively discover that vim-common requires vim-filesystem, glibc and bash which in turn requires filesystem, glibc and ncurses-libs.
You can go up or down at any point. For example the setup package is a requirement of filesystem, going again up in the dependency tree, reveals that filesystem is required by basesystem, filesystem (as we already saw), initscripts, rpcbind and shadow-utils.
Parameterized Debug Builds for Rust Packages
As a last point I want to leave you with a little something I adopted for packaging Rust applications, which sometimes takes an inhuman amount of time to build in --release mode.
Usually I define the following macro in the spec file, so that when I need a fast build to verify something at runtime, it will trigger cargo build instead of cargo build --release and save some tens of minutes of build time.
%define build_target_dir target/release/
%if 0%{?debugbuild:1}
%define build_target_dir target/debug/
%endif
[...]
%build
%if 0%{?debugbuild:1}
cargo build
%else
cargo build --release
%endif
%install
[...]
install -p -m 755 ./%{build_target_dir}/%{name} %{buildroot}/%{_bindir}/
When rpmbuild is called with the --define 'debugbuild 1' flag like so:rpmbuild --define 'debugbuild 1' -bb myapp.spec
this will correctly build and deploy the binary in both cases.
If you want to verify how it works, you can use the rpmspec -P command from earlier, as it also takes --define flags like so:rpmspec --define 'debugbuild 1' -P myapp.spec
The result will be something like this with debugbuild set to 1:
[...]
%build
cargo build
%install
[...]
install -p -m 755 ./target/debug/myapp /home/benjamin/rpmbuild/BUILDROOT/myapp-0.0.1-1.x86_64/usr/bin/myapp
As you can see, the build is correctly invoked without the --release flag, and the installed binary is taken from the debug target.
Benjamin Gröber
R&D Software Architect at Wuerth Phoenix
Hi, my name is Benjamin, and I'm Software Architect in the Research & Development Team of the "IT System & Service Management Solutions" Business Unit of Würth Phoenix.
I discovered my passion for Computers and Technology when I was 7 and got my first PC. Just using computers and playing games was never enough for me, so just a few months later, started learning Visual Basic and entered the world of Software Development. Since then, my passion is keeping up with the short-lived, fast-paced, ever-evolving IT world and exploring new technologies, eventually trying to put them to good use. I'm a strong advocate for writing maintainable software, and lately I'm investing most of my free time in the exploration of the emerging Rust programming language.
Author
Latest posts by Benjamin Gröber
25. 11. 2022
Development
What is the perfect development team size?