Sunday, August 12, 2007

Improving patches management at distro level

Federico, your remarks about patch management for distributions are quite similar to a mail I sent two years on Mandriva maintainers list, about my dream process for patch management. At that time, Git wasn't really on the radar but I thought it could be interesting to share it with everybody, now we might have tools to help :

Here are some ideas I've been having on how we could improve maintainer workflow concerning patches integration (which is quite critical for GNOME or KDE, for instance). I've been influenced by Mark Shuttleworth keynote about some stuff Ubuntu is working on (I was refering to Mark keynote at GUADEC 2005, about Launchpad).

Most of the time, maintainers are applying various patches on upstream source to generate its packages. For a good part, patches are extracted from CVS/SVN/whatever versioning system is used by upstream software maintainers, either in HEAD branch or other branches. Other potential patch sources are other distributions packages, upstream bugzilla or maintainers own brain.

Extracting patches from various CVS can be quite time consuming (this has improved with SVN since the original writing) and keeping history of patches, when upstream partially merged them is very time consuming.

When patches are written by maintainers, upstream integration is usually slow because it requires either submitting bugs upstream with patches or committing patches on upstream CVS once review process has approved it.

In order to improve this, maintainers could use a versioning system and/or magic scripts doing the following :

  • upstream revisions and branches would be accessible in the repository just like using upstream repository
  • Mandriva patched source would be using a separate branch containing source with all patches applied. Each patch could be done applied in a separate sub-branch, Mandriva top branch would merge all those sub-branches (this last point might not be that useful, since we are already using "Repository System" for building packages.
  • When Mandriva patches are supposed to be merged upstream, branch relevant for each patch could be published for upstream integration or committed directly on upstream repository if approved by upstream maintainer.
  • When fixes are needed for Mandriva package and are already available upstream, merging from upstream branch to a Mandriva sub-branch would ease package maintainer job, while keeping history and origin of the fix.
  • When new upstream release is made available, it is merged into Mandriva branches and if some patches were merged upstream, they are removed automatically, since revision history is kept in each branch and the versioning system is supposed to be smart enough to deal with that.
In order to integrate easily when new Repository Build system, there might be some scripts magic to convert Mandriva patches sub-branches into subversion Build Repository (I'm refering here to Mandriva build system which is SVN based).

In an ideal world, each distribution could have their own branches and maintainers from each distribution would be able to access another distribution published branches, to easily find patches. And we would have a free (as speech) way to have a complete view of bug reports and available patches :) And I'm not even talking about migrations of bug between bugzilla (but at least, with Mandriva switch to Bugzilla 3, we will be ready for the future).

(end of mail, italic are comments I added today)

Comments welcome ;)

4 comments:

  1. Comment from Adam Williamson:

    "a mail I sent two years (ago)"

    "but at least, Mandriva switch to Bugzilla 3 will be ready for the future"

    good to see that happened on schedule...=)

    ReplyDelete
  2. I've just fixed missing italic for the bugzilla 3 part.

    ReplyDelete
  3. Comment from Ken VanDine:

    Sounds like a problem that was solved already by conary (solved by the package manager mostly). Check it out, http://wiki.rpath.com/wiki/Conary. Conary is the package manager that is used in Foresight Linux, I would love to see Mandriva (and others) move in that direction as well.

    ReplyDelete
  4. Not really. Conary is more a way to easily created derivated distributions (with branches and so one). It does store all source code used to create distributions in SCM, but that would help a lot for the problems I'm interested to solve. Moreover, it requires to change packaging entirely (which is not an option for most distributions).

    ReplyDelete