HOWTO: Backport and host Debian & Ubuntu packages on the OBS

[Stevo]

I copied this from my Debian post. Over here on MX, we've gone way past the point of being able to do a smooth upgrade to a new stable version when it comes out, so all that "~" stuff goes out the window if you're just doing your own ports, or don't care about Debian users. Just slap the source files direct from upstream into the OBS and see what happens.

(Part 1 of 3)

* NOTE: as explained below, more and more backports are requiring a newer debhelper
and dh-autoreconf in each repository (project) we create. We can cut down on the
work by backporting everything we do in one project, so they all use the backported
debhelper and dh-autoreconf we've already added to the project.

The alternative is to just remove the need for those newer build tools. This can usually
be done by commenting out the two lines in the override section for dh_strip in /debian/rules,
and dropping the minimum required version of the debhelper build-depend in debian/control
if necessary (often the maintainer forgets to bump the B-D) We have the choice--whatever's
the least trouble.

OBS = openSUSE Build Service: https://build.opensuse.org/

The free and easy to join openSUSE Build Service builds and hosts both Linux rpm
and deb packages from source packages. This guide focuses on building packages
for Debian and Ubuntu (deb format).

Prerequisites: an account at the OBS, which is easy to create via their web
interface. Once we're logged in, currently a cookie will keep us logged in for 24
hours, or until we log out.
Have the "devscripts" package installed on our machine.
Source packages to load into the OBS.

Source packages can be obtained from upstream Debian, Ubuntu, PPAs, or other
repositories as described here: https://wiki.debian.org/SimpleBackportCreation
If we don't plan to actually build any packages on our own machine, we should
be able to get by with just an install of devscripts instead of the bigger
packaging-dev metapackage. Source packages can also be directly downloaded to
our machine from packages.debian.org,



or surfing down into the package pool in a PPA, for example. This can be quicker
than modifying our software sources,downloading, then reverting the changes.

Often, the upstream Debian source packages can be directly uploaded into the OBS
without changes. However, if we, like the official backports crew, are concerned
with allowing users a smooth upgrade in place to the next Debian release, adding a
tilde "~" symbol allows our version to be seen as a lower version than the same
one in the new stable. So if the upstream version is

foo-1.2.3-1

at the time of the stable transition, changing that to

foo 1.2.3-1~

will make that a lower version than the first, and the user will be able to upgrade
smoothly to the new Debian stable version. Jessie-backports adds "~bpo80+1, and
ordinarily we'd be able to upload those source packages directly, but
unfortunately, the OBS does not recognize the "+" symbol, so we have to either
add a new debian/changelog stanza with a modified version (~bpo80.1), per the wiki
process above, or directly edit the version string in the existing first stanza.
I'll do that below in my example--what's important is the "~" to allow the
upgrade process. If we aren't concerned about a smooth upgrade, perhaps if the
package isn't in upstream Debian and probably won't be in the next release, then
this isn't a problem.

The OBS can build for multiple Debian and Ubuntu releases from one set of source
packages, such as in my Pale Moon repository, so in that case, the version
shouldn't be distro-specific. Since PM isn't in those distros, I just use a version
like "26.5.0-1" in the changelog. Having distro-specific versioning, such as
"26.5.0-1bpo80.1", would require having a separate subproject repository (see below)
for each distro with its own specific source packages, plus a single distro build
target for each subproject. That's not a problem if we're just doing backports
for one release, of course.

In addition to not allowing the "+" symbol, the virtual machines that the OBS
provides have limited RAM. Almost all packages will build, but memory-hungry
ones that eat a lot of RAM during the build, such as llvm or Chromium, will fail.
Some packages may also just get a random build failure that is usually cured by
triggering a rebuild. ¯\_(ツ)_/¯ ??

The OBS will also trigger a rebuild of everything in a repository if you upgrade
one of the base build-dependencies. For example, I have a multimedia repository in
which most of the packages depend on the latest ffmpeg release. Upgrading ffmpeg
also ends up triggering a rebuild of most packages. Upgrading an ffmpeg build-
depend such as libx265-dev also triggers a ffmpeg rebuild and the rest of the
cascade. Though this might take a few hours to complete, this is actually more
of a feature, since it ensures that all those multimedia packages remain compatible
across soname bumps, which can cause breakages deb-multimedia and even jessie-
backports have had trouble avoiding.

So, we created a new account on the OBS. We automatically get assigned a "home
project". For my example below, since I already have Pale Moon in my home project's
repository, I'll create a separate subproject so the repositories remain separate.
So, since we'll modifying the source, and this is a modern 3.0 (quilt) source
format package, we only need to download the two highlighted files shown in the
previous image. Then the two tarballs are extracted:



and the version in the first stanza of the /debian/changelog file is modified,
per the wiki backporting guide. If we're really following Debian policy exactly,
we go through the procedure to add a new stanza, but for the OBS, we can get away
with just adding something like ~bpo80.obs to the existing version. In this case,
I already added a new stanza for the MX 15 backport, so I'm just going to edit that:





Like before, the "~" is often the most important symbol. If you don't care about that,
just try downloading and uploading all the Debian source packages (usually three).
Also note that the build tools are really picky about syntax in this file, so if
we accidentally delete one of two whitespaces between words in certain places in
this file, the build system can throw a fit. Other places, it doesn't care. Safest to
just not change spacing, just carefully change words.

(continued)

[Stevo]

(Part 2 of 3)

So once our changes are saved to the changelog, we put the /debian folder into the
extracted source folder, then enter the source folder, open a terminal there, and
issue:

Code: Select all

debuild -S -uc -us

This regenerates a new debian tarball and a .dsc digital hash file. Since we changed
the contents of /debian, the hashes changed, we need new files. The orig tarball
should never change, so if it's a big one, we could be uploading it to the OBS to
save some time while we work on the others.

Anyway, here's my home project page, already occupied by Pale Moon. I don't want
any users of that repo to be confused with a new Gnumeric, so I'll go to my Subprojects
and create a new one for Gnumeric.





So I hit the (3) link for a new one.

Give it a name--Title and Description are optional, and hit "Create Project".



Success!



Now we need to add target repositories for our project, so hit the Repositories
link. In this screen, checking the box for a repository will automatically add it,
and we'll get a success message at the top. We're just going to add Jessie; repositories
can also be added and deleted later.

We can navigate around by clicking on the breadcrumb set of links, most of which
are visible at the top of the above graphic, so clicking on "Gnumeric" takes us to the
subproject's "root".



Our source files get uploaded into a package, so we have to create one for
gnumeric. It's important that we use the actual name of a package that gets built
as listed in the /debian/control file. This makes it easy to publish our repository.
We only need one actual package name that will appear in our repo to publish it,
though--so gnumeric is good enough, and we could be more lax with the other four
packages it turns out we need to add later. So projects can have any name, but we'll
need to make sure one actual package name exists in that project.

Creating a gnumeric package, again anything besides a name is nice but optional:



Success:



(continued)

[Stevo]

Part 3 of 3

Now we need to upload our source files for the package, so click on "Add file",
then choose a local file (the orig.tar.xz for gnumeric in this case)



and "Save Changes" to upload it. Chromium-based browsers can display a % progress of
the file upload at the bottom, I haven't seen it in Firefox, though.



Once uploaded, eventually OBS will display success and allow the next source file
to be uploaded.



Once all are loaded, we give OBS a minute or two, then refresh the page to see the
build status.



Oh, no! Standard Jessie is missing something needed to build this package. Mouse
over the status message on the right to see what.



So we also need to backport newer versions of libgoffice-0.10-dev and libgsf-1-dev,
using the same procedure, and load their sources into new packages for our project.
Then the gnumeric build will recognize and use the new versions once they are complete.

So one of those new builds fails eventually anyway. Clicking on the "failed"
message brings up the build log with the error:



This means that we need the debhelper from jessie-backports, and that the Debian
maintainer forgot to list the new minimum required version in the build-depends.
Another unresolvable then shows we need dh-autoreconf 12 from backports, also.
So we get the tar.xz source files for each from jessie-backports (just one for each,
they're native Debian packages, so include a debian folder already) Ordinarily, we
could just upload their tarballs and dsc file to the OBS, but we have to get rid of the
"+" in their "~bpo80+1" version string and make it "bpo80.1", again in the first
changelog stanza, and then regenerate new source files. In this case, we have to
upload the new tar.xz and .dsc file we made for each package. If you plan any more
backport projects on the OBS, keep the sources for these two--you'll need them.

After everything, success.



Note that I didn't bother to give the newer debhelper or dh-autoreconf packages
the proper names, though anyone that adds our new repository will see the newer correct
versions from the debian/control file. They won't need to upgrade those two, though--
those are only build tools. Upgrading gnumeric will automatically pull in the newer
gsf and goffice packages, though.

Click on the gnumeric package link, since that's the one with the correct package
name.



The repository is automatically created and a key created. We just need to give out
the "Download package" link, which has the instructions for adding the repo and the
key.

[v3g4n]

Spectacular and can't wait to give it a good read! Thanks!!

[mmikeinsantarosa]

Nice job OWO

[Stevo]

I have to add a section on how to remove files so the corrected ones can be added if there's a problem, or you want to update the whole package--such as gnumeric-1.12.32 to -1.12.33. You just hover the mouse over that red spot to the right of each file name. If you have to edit the debian folder to fix a build failure, the source files must be regenerated and loaded, but the .orig tarball can stay on the OBS, as it never changes. That's nice because it's always the biggest.

All files must be deleted and updated for a source package upgrade, of course.

[Paul..]

...a beautiful thing...