Rework Debian packaging around system dependencies - #3019
Conversation
Make Debian release packages follow Debian dependency and binary-package conventions while preserving the existing self-contained CMake superbuild for other platforms and users. Build common dependencies from distro packages when MAVSDK_USE_SYSTEM_COMMON_DEPENDENCIES is enabled: Asio, curl, fmt, liblzma, nlohmann-json, and TinyXML2. Continue fetching pinned MAVLink, libevents, PicoSHA2, and libmavlike sources because Debian does not package them, and document the resulting Debian archive source constraints. Use native Python and python3-lxml in Debian builds and skip only MAVLink private pip bootstrapping there. Keep the legacy pip-backed behavior enabled by default for ordinary superbuild users. Split the output into libmavsdk3 runtime and libmavsdk-dev development packages, with debhelper-generated detached debug symbols. Derive runtime dependencies through dpkg-shlibdeps, install libraries in multiarch paths, propagate cross tools through nested superbuilds, and correct static pkg-config private dependency metadata. Add Debian 12 and 13 amd64/arm64 package CI, leave Debian 11 clearly labeled on the legacy path, and prevent the old fpm arm64 job from publishing conflicting packages. Keep debian/changelog and release versions maintainer-owned, checking tagged releases instead of rewriting them. Work around Debian libpsl bug #1111197 only for the Debian 13 amd64-to-arm64 libpsl-dev transaction using --force-overwrite, with an explicit removal note. Leave debian/copyright unchanged for maintainer review. Validated with Debian 13 amd64 and arm64 binary builds, Lintian apart from the intentionally excluded existing copyright-template finding, an extracted arm64 CMake consumer, static pkg-config resolution, and the default bundled CMake superbuild plus installed native consumer.
Configure libmavlike only after the common-dependency block so self-contained superbuilds install bundled TinyXML2 before libmavlike searches for it. System-dependency builds continue using their packaged TinyXML2. Validated with a fresh default superbuild configure while explicitly hiding the host TinyXML2 CMake package.
|
Temporarily closing this draft while the corrected head is validated in fork CI. I will reopen it once the full matrix is in a reviewable state. |
Use MAVLINK_INSTALL_PYTHON_DEPENDENCIES=OFF for iOS just as for system-dependency builds. The main MAVLink patch now owns conditional Python setup, so remove the overlapping legacy iOS patch that no longer applies after the main patch. This keeps iOS builds from bootstrapping Python packages while avoiding two patches that modify the same upstream CMake sections. Validated by running the IOS ExternalProject path against the pinned MAVLink revision, applying the unified patch, generating headers without pip bootstrap, and installing successfully.
| os_version: "24.04" | ||
| - os_type: debian | ||
| os_name: Debian | ||
| os_name: Debian (legacy) |
There was a problem hiding this comment.
LTS support for Debian 11 is about to run out. I wonder if we should drop this one.
There was a problem hiding this comment.
I anticipated dropping it, but I left it in for the CI run. EOL is at the end of August, and I would say most people have already moved on.
| * Build common dependencies from Debian packages. | ||
| * Split the runtime library from the development files. | ||
| * Generate runtime dependencies from shared-library metadata. |
There was a problem hiding this comment.
This is going to be a pain and forever conflict. We use the version number from the git tag, so we try not to have it in source itself. Could we just link to the github releases or something like that?
There was a problem hiding this comment.
I was thinking of a few different ways to do this. In CMake we could use FetchContent_{Declare, Populate, GetProperties} with specific git tags or hashes. There's also The GIT_URL and associated features inside ExternalProject_Add, that will essentially accomplish the same thing as the superbuild, fetching the source from a specific hash or tag.
There was a problem hiding this comment.
We already get the git tag in cmake:
Lines 18 to 41 in 99f6c9b
So, we should probably make this file a template and fill it out with cmake?
| Section: libs | ||
| Homepage: https://mavsdk.mavlink.io/main/en/index.html | ||
| Priority: optional | ||
| Maintainer: MAVSDK Development Team <julian@oes.ch> |
There was a problem hiding this comment.
Maintainer: fullname-email (recommended)
Should be in the format “Joe Bloggs jbloggs@foo.com”, and is
typically the person who created the package, as opposed to
the author of the software that was packaged.
So this would be you, not me, right?
There was a problem hiding this comment.
Well I did fill it out with your name :). In all seriousness, I think if we can get this set up right, it won't take much maintenance. I wouldn't care to help out in this regard though.
There was a problem hiding this comment.
I expect this to be my responsibility, so yeah my name is probably right. That being said, the spec says it would be you as you're adding this...not sure how much it matters - probably best if it's just me to keep it consistent indeed.
| if(NOT IOS) | ||
| build_target(zlib-ng) | ||
| endif() | ||
| if(NOT MAVSDK_USE_SYSTEM_COMMON_DEPENDENCIES) |
There was a problem hiding this comment.
A bit ugly but I see what you're doing. To me this should just be "not-superbuild", and then you have to run some script to prepare the things missing.
There was a problem hiding this comment.
I had thought about that, but then I realized, some people may not be on debian, and also not want to do the superbuild, etc.. I know it's probably an edge case, but I know many people use red hat and its derivatives. I suppose I could also look towards packaging for those OS's in the same way, but that would certainly be more code that is new, and less that is changed.
There was a problem hiding this comment.
What about a script for Debian that installs/prepare things as required, like the one we had, and then one for Fedora/etc. We used to ship rpms but no one seemed to need/want them, so I dropped it.
I just don't want 3+ ways to build it. It should be either:
- SUPERBUILD - everything taken care of
- not SUPERBUILD - you do whatever you want, in this case grab packages required and install the rest via the system.
If we want to make sure the two ways are not getting out of sync, what about adding a file with env vars that lists the git hashes or tags per repo and then use those strings in either the cmake SUPERBUILD or the debian package tooling script?
|
I wanted to get this out for you guys to review. I'm open to any comments or suggestions that make it work better with your flow, or anticipated changes. There's a lot of ways to make it work, and the one thing I don't want to do is add more work for you both. There's also the matter of the four sources that are still built inline. If you want to take this further, after testing this out, I could make those into their own debian packages as well, and work towards a true debian stack of packages and dependencies. Edit: There are a few questions I have about the other dependencies, but I'd rather focus on one improvement at a time. |
Summary
MAVSDK already has a useful Debian release path alongside its self-contained CMake superbuild. This PR carries that direction through to Debian's normal packaging model: common libraries come from the distribution, package metadata describes the resulting dependencies, and the existing bundled build remains the default everywhere else.
Packaging changes
MAVSDK_USE_SYSTEM_COMMON_DEPENDENCIESpath for Asio, curl, fmt, liblzma, nlohmann-json, and TinyXML2. Ordinary CMake builds remain unchanged.dpkg-shlibdeps, and install into the correct multiarch paths.libmavsdk3,libmavsdk-dev, and detachedlibmavsdk3-dbgsympackages, with corrected CMake/pkg-config metadata for consumers.The remaining fetched sources mean this is suitable for MAVSDK's upstream binary releases, but not yet for Debian archive upload.
debian/README.sourcerecords that boundary. The existing licensing file is deliberately left for maintainer review.Validation
The complete project matrix passed on the exact PR head,
b2ca97dc5:This also exercises the default bundled superbuild, system-dependency builds, CMake consumers, cross-compilation, package installation and contents, Lintian, static pkg-config resolution, and separated debug symbols.