Plugin Lifecycle Andrew Fabian MetaArchive Annual Membership Meeting Atlanta, Georgia Friday October 24, 2008

Plugin Lifecycle START Local Development and Testing Commit to Repository New Collections In Conspectus Isolation Testing Plug-in Modifications Conspectus Modifications Production! END

Local Development Note: There is a presentation on this topic by Daniel Culpepper. Write the Plugin with LOCKSS Plugin Tool. Test-crawl with that tool. – Crawling will happen from your desktop computer, not your LOCKSS box. Make sure firewalls are OK with this. This implies setting up a Manifest page on the site to be crawled. When finished you have an.XML Plug-in file.

Subversion Subversion synchronizes files between MetaArchive developers. You’ll need a Subversion client. – TortoiseSVN for Windows – SCPlugin for Macs – svn (command-line) for everything After a plugin is written, add it to the MetaArchive repository. Releases (testing and production) are made from the official repository only.

The Conspectus Database This is the MetaArchive’s online master list of what gets harvested. It contains: – Title – Descriptive Metadata – Base URL – Plugin Identifier – Extra Parameters Add a new record for the collection you’re getting ready to harvest. – These are fairly detailed records. Make sure to have the appropriate descriptive metadata on hand.

The Conspectus Database

Isolation Testing This tests two things: – The final combination of the Plugin, Collection, and production software – Recrawls MetaArchive Sandbox: – VMWare Virtual Machine – Gets Plugins from Subversion, Collections from the Conspectus – Runs the MetaArchive LOCKSS daemon, but does not communicate with the rest of the network

Isolation Testing Demo

Going Live Almost the same process as Isolation Testing Ask MetaArchive admins to release the plugin and the collection on a weekly phone call. They will install the plugin for the entire network, and allocate space on a set of MetaArchive storage nodes to cache the new collection.

Plugin Modifications In Production Go through normal development cycle Make sure to increment the Plugin Version Number! Ask admins to update the Plugin from Subversion – All MetaArchive caches will load the new plugin within a day. – For faster updates, trigger a manual crawl of the “Plugins” AU from the DebugPanel.

Conspectus Modifications In Production Edit Descriptive Metadata at any time Add additional AUs at any time – Add a new line to the “Extra Parameters” section – Announce the new AU via and weekly phone call Changing existing AU parameters can break things. Instead, add a new AU and then delete the old.

Deletion If a collection no longer needs to be harvested, and the backup copy is no longer useful, it can be deleted. Deletion is tricky, because there are several steps: – Delete Collection from Conspectus – Delete AUs from all participating MetaArchive Caches – Delete files from disk of all participating MetaArchive caches Detailed instructions are in the MetaWiki.

Questions? Answers – Yes – fixed in next version – No – 42 – legacy support – it’s not a bug, it’s a feature