Managing 666 packages, or how to tame the beast Introducing PET, a tool to track packages' health

What ● Finding out what packages need love ● Gather information in one single place ● Be able to “see” the overall situation ● Make team work easier ● Track packages in the archive and in VCS

Existing tools

Way too much

Shortcomings ● Good information, but dispersed ● Lack of categorisation, filtering, sorting, etc. ● Doesn't track work in progress ● Grouping key: maintainer ● Non-DDs need to RFS

A different approach ● Grouping key: the VCS repository ● Update information on every commit ● A few conventions about repository usage ● Don't make reports, build a database to report from ● Bonus points: make it modular, separate the presentation from the logic

PET Package Entropy Tracker ● Started as a few ad-hoc scripts in pkg-perl ● Rewritten during 2007 for better code and wider audience ● Not only me, Damyan Ivanov and Gregor Herrmann

Not only for packaging groups! Also useful to keep track of your own packages

Sources of information ● Debian archive (source packages) ● SVN (to be extended to other VCSs) ● BTS ● Upstream websites (watch files) ● New ideas? – Popcon – DEHS – svn-buildstat, etc.

Archive information ● Current versions in all distributions – Flag packages that aren't in unstable, and versions that disappeared after an upload ● Control data: – Detect packages that don't have the group as Maintainer or Uploader. – DM-Upload-Allowed status – Map source and binary packages ● Presence in NEW and INCOMING

Repository information ● Same information as the archive (reading debian/control) – But current ● Know which are in fact all your packages ● Presence of patches ● Changelogs ● Watch files ● Differentiate work in progress from released work (“UNRELEASED” distribution, tags)

Conventions ● Assume a few conventions are followed ● Encourage good practices in the way ● Use the changelog as a communication medium among team members – TODO notes, warnings to other members – Use the UNRELEASED tag to differentiate ready to be uploaded (or already uploaded) packages ● Create a tag on each upload

Watch files ● Is there a new upstream version? ● Included in the price: an uscan equivalent, written from scratch. – Mental note: release it as uscan-ng – Mental note 2: design a sane watch file format ● Perl group bias: use CPAN indexes to avoid most HTTP queries

CGI report goodies

Packaging workflow Nothing new here, just the assumed workflow ● Do initial import / merge new upstream version – You already know you want all source changes in separate patches – Set the distribution to “UNRELEASED” ● Commit early, commit often – You end writing meaningful changelogs in the way – Other members can see what you're up to

Packaging workflow ● When you're done – debchange -r (sets the distribution to “unstable”) – debcommit ● Upload it and tag it – debcommt -r ● Or wait for a DD to do it for you. No need to ask, as your package already shows as “ready for upload”

Components ● Data retrieval script – Run on each commit – Long running tasks done in a cronjob – Knows data relationships (watch file updated implies need to rescan upstream) – Uses Storable files as backend

Components ● Reporting tools – CGI (qareport.cgi) – RSS feed with new packages ready to be uploaded – CLI (qareport) – Custom reports (CPANTS metrics)

Future ● Remove SVN dependency, make the VCS component pluggable ● Allow meta-repositories to be created spanning several sources ● Give proper structure the internal data model and the backend database (Perl hashes make lazy programmers) ● Integrate other sources of information

Questions?