Plugin Lifecycle
Andrew FabianMetaArchive Annual Membership
MeetingAtlanta, Georgia
Friday October 24, 2008
Plugin Lifecycle
START
Local Development and Testing
Commit toRepository
New CollectionsIn Conspectus
IsolationTesting
Plug-inModifications
ConspectusModifications
Production! END
Local Development
Note: There is a presentation on this topic by Daniel Culpepper.
• Write the Plugin with LOCKSS Plugin Tool.
• Test-crawl with that tool.– Crawling will happen from your desktop computer,
not your LOCKSS box. Make sure firewalls are OK with this.
• This implies setting up a Manifest page on the site to be crawled.
• When finished you have an .XML Plug-in file.
Subversion
• Subversion synchronizes files between MetaArchive developers.
• You’ll need a Subversion client.– TortoiseSVN for Windows– SCPlugin for Macs– svn (command-line) for everything
• After a plugin is written, add it to the MetaArchive repository.
• Releases (testing and production) are made from the official repository only.
The Conspectus Database• This is the MetaArchive’s online master list of
what gets harvested.
• It contains:– Title– Descriptive Metadata– Base URL– Plugin Identifier– Extra Parameters
• Add a new record for the collection you’re getting ready to harvest.– These are fairly detailed records. Make sure to have the
appropriate descriptive metadata on hand.
Isolation Testing
• This tests two things:– The final combination of the Plugin, Collection,
and production software– Recrawls
• MetaArchive Sandbox:– VMWare Virtual Machine– Gets Plugins from Subversion, Collections from
the Conspectus– Runs the MetaArchive LOCKSS daemon, but
does not communicate with the rest of the network
Going Live
• Almost the same process as Isolation Testing
• Ask MetaArchive admins to release the plugin and the collection on a weekly phone call.
• They will install the plugin for the entire network, and allocate space on a set of MetaArchive storage nodes to cache the new collection.
Plugin ModificationsIn Production
• Go through normal development cycle
• Make sure to increment the Plugin Version Number!
• Ask admins to update the Plugin from Subversion– All MetaArchive caches will load the new plugin within a day.– For faster updates, trigger a manual crawl of the “Plugins”
AU from the DebugPanel.
Conspectus ModificationsIn Production• Edit Descriptive Metadata at any time
• Add additional AUs at any time– Add a new line to the “Extra Parameters”
section– Announce the new AU via email and weekly
phone call
• Changing existing AU parameters can break things. Instead, add a new AU and then delete the old.
Deletion
• If a collection no longer needs to be harvested, and the backup copy is no longer useful, it can be deleted.
• Deletion is tricky, because there are several steps:– Delete Collection from Conspectus– Delete AUs from all participating MetaArchive Caches– Delete files from disk of all participating MetaArchive
caches
• Detailed instructions are in the MetaWiki.