1. Managing 666 packages, or how to tame the beast Introducing PET, a tool to track packages' health

2. What ● Finding out what packages need love ● Gather information in one single place ● Be able to “see” the overall situation ● Make team work easier ● Track packages in the archive and in VCS

3. Existing tools

4. Way too much

5. Shortcomings ● Good information, but dispersed ● Lack of categorisation, filtering, sorting, etc. ● Doesn't track work in progress ● Grouping key: maintainer email ● Non-DDs need to RFS

6. A different approach ● Grouping key: the VCS repository ● Update information on every commit ● A few conventions about repository usage ● Don't make reports, build a database to report from ● Bonus points: make it modular, separate the presentation from the logic

7. PET Package Entropy Tracker ● Started as a few ad-hoc scripts in pkg-perl ● Rewritten during 2007 for better code and wider audience ● Not only me, Damyan Ivanov and Gregor Herrmann

10. Not only for packaging groups! Also useful to keep track of your own packages

11. Sources of information ● Debian archive (source packages) ● SVN (to be extended to other VCSs) ● BTS ● Upstream websites (watch files) ● New ideas? – Popcon – DEHS – svn-buildstat, etc.

12. Archive information ● Current versions in all distributions – Flag packages that aren't in unstable, and versions that disappeared after an upload ● Control data: – Detect packages that don't have the group as Maintainer or Uploader. – DM-Upload-Allowed status – Map source and binary packages ● Presence in NEW and INCOMING

13. Repository information ● Same information as the archive (reading debian/control) – But current ● Know which are in fact all your packages ● Presence of patches ● Changelogs ● Watch files ● Differentiate work in progress from released work (“UNRELEASED” distribution, tags)

14. Conventions ● Assume a few conventions are followed ● Encourage good practices in the way ● Use the changelog as a communication medium among team members – TODO notes, warnings to other members – Use the UNRELEASED tag to differentiate ready to be uploaded (or already uploaded) packages ● Create a tag on each upload

15. Watch files ● Is there a new upstream version? ● Included in the price: an uscan equivalent, written from scratch. – Mental note: release it as uscan-ng – Mental note 2: design a sane watch file format ● Perl group bias: use CPAN indexes to avoid most HTTP queries

16. CGI report goodies

23. Packaging workflow Nothing new here, just the assumed workflow ● Do initial import / merge new upstream version – You already know you want all source changes in separate patches – Set the distribution to “UNRELEASED” ● Commit early, commit often – You end writing meaningful changelogs in the way – Other members can see what you're up to

24. Packaging workflow ● When you're done – debchange -r (sets the distribution to “unstable”) – debcommit ● Upload it and tag it – debcommt -r ● Or wait for a DD to do it for you. No need to ask, as your package already shows as “ready for upload”

25. Components ● Data retrieval script – Run on each commit – Long running tasks done in a cronjob – Knows data relationships (watch file updated implies need to rescan upstream) – Uses Storable files as backend

26. Components ● Reporting tools – CGI (qareport.cgi) – RSS feed with new packages ready to be uploaded – CLI (qareport) – Custom reports (CPANTS metrics)

27. Future ● Remove SVN dependency, make the VCS component pluggable ● Allow meta-repositories to be created spanning several sources ● Give proper structure the internal data model and the backend database (Perl hashes make lazy programmers) ● Integrate other sources of information

28. Questions?